Serdar,
First, what are we talking about? a free standing uC with noise generated on the board or noise from the peripherals.
Second, how does the noise affect you, sometimes that is a hint as to the source.
Third do you have capacitors 10-100nF across every chip on your board?
Fourth do you have a 4.7-47 uF tantalum across the '51

Erik