Hallo people,

Whoops! Isn't it possible to have some discussion without people becoming angry so rapidly?

If ground is referenced to some point inside rectifier loop, then voltage drop across connection between big elko and this ground reference point is superimposed to output voltage of 5V regulator. That's just physics, and I'm sorry when this offends people.
Why such voltage drop is critical?
Assume you have 500mA current flowing out of regulator. Then, for keeping voltage at elko at the same level a much more bigger current has to flow when rectifier charges up elko. Assume that at only 1/20 of mains period rectifier opens, then 20 times 500mA will flow. So, up to 10A can flow accross any connection inside rectifier loop.
Situation is becoming even worse when transients on mains are present at transfromer input. Then also inductance of this connection can make trouble.
I have seen many times, that supply voltage generation did suffer from right this issue. And there many people in EMI trouble shooting companies working on just to fix applications from such traps.
Shouldn't be such a forum the right place to discuss this topic, just when so many members are suffering from time to time from 'unknown' and 'strange' interference and try to solve problems with varistors and optocouplers, fruitlessly?

Why are capacitors accross rectifier diodes recommended?
Of course, in normal operation (means with clean mains) such capacitors are completely without any sense. But clean mains is not representing real life. Assume that common mode transients of about 2kV (2000V!) are hitting primary winding of transformer. Don't think that they are seldom, they are to be expected occuring many times a day! 'Common mode' means that they are referenced to earth. When you connect power supply to outer wolrd there is allmost anywhere some referencing to earth, at least via soft grounding capacitances of about 10nF.
Problem now arises from interwinding capacitiy of transformer which transfers dangerous voltage directly to secondary winding. And where do you think overvoltage makes trouble? Right, at rectifier diodes! Not at transformer itself, because normally it is designed to withstand these high voltage trasients.
Interwinding capacitance is about 200...300pF, for low power transformers. Rectifier diodes normally have a junction capacitance of less than 50pF, when they are low current types. At high reverse voltage even less than 10pF. So, when a transient of 2kV is hitting transformer for a short amount of time rectifier diodes have to withstand extremely high overvoltages. And it's just the low current types which suffer most, because higher current types have more junction capacitance.
If now 10nF decoupling capacitors are connected accross rectifier diodes, overvoltage is highly reduced, up to only 1/50 of transient. That's a range a rectifier can live with, without problems.
Decoupling capacitor has another big advantage: When earth connection is seeing some longer path, inductivity of this connection plays a role. Then decoupling capacitor accross rectifier diode drastically reduces resonances which also can lead to overvoltages in combination with softer transients.

Now, I can hear people saying, that high overvoltages cannot be generated because forward biased rectifier diode would clamp it. That might be true for (expensive, and so not wall wart relevant) fast recovery diodes. But normal ones would need many, many microseconds before conducting and consequently they are also in danger to be destroyed.

Modern electronics use both, fast recovery diodes in combination with decoupling capacitors accross diodes. But that you will not find in a wall wart power supply, of course... Never wondered about why they become damaged from time to time? Ah, almost forgotten, 2kV transients is not upper limit!

So, before becoming angry have a look at modern design practices and modern CE standards.

Bye,
Kai