You cannot stick the 'S51 into the SAME positions on the breadboard than the '2051, do you?

Simply, those positions where the '2051 is, are worn out, or something.

The big decoupling cap on the power supply is worhtless for shorting out the high frequency noise which is created by the running micro - and that's that noise which kicks the same micro to death. You need a ceramic cap (or, better, two, 100nF and 100pF-1nF in parallel - both have different high-frequency behaviour) as close to the pins as possible (to decrease parasitic inductance of the leads, which would eliminate the effect of the caps on HF).

If you are not shy to post a picture of your setup (read http://www.8052.com/faqs.phtml?FAQ=120199 and the "noob" document - link in the FAQ) we could point out perhaps some of the errors you might be making.

JW