Do you have a ground loop? Well: read what I wrote here, get a good understanding of what a ground loop is, note that some components besides wires are short circuits for AC, examine your situation again and decide for yourself!

A ground loop means that what you think is an earth wire is actually part of a loop, to which you are oblivious.
Ground loops are for instance formed when two pieces of equipment which are both earthed are connected together using a screened cable. Mind you: this is NOT the only way to have an earth loop but it is by far the most common and illustrative. If you do this, you get an instant ground loop. It is a PHYSICAL loop with a BIG HOLE in it. And if you have ever learned about induction, you will know that electromagnetic disturbances will induce currents in a closed conducting loop. So if there is a fridge nearby, it may cause nasty things to happen.
Now WHY is this a problem? WHY does a ground loop cause problems? If the loop only consisted of earth and the cable screen, there wouldn't be much trouble. But usually, the path of the ground inside a piece of equipment (and particularly inside equipment which wasn't properly designed!) usually from the mains inlet ground lug, across the PCB past all the components to the screen connection for the screened cable. So in your ground loop, there is the PCB ground track, unless you damn well knew what you were doing when you designed the thing... And here comes the nasty bit: this PCB track has a relatively high resistance compared to the other parts of the earth loop. So if a current is induced in the ground loop by a nearby magnetic field, a relatively large voltage drop will form across your PCB ground track compared to the rest of the loop. And since the PCB ground track, particularly in poor designs, meanders from component to component, there will be a DIFFERENCE in ground potential for every component in the path. Especially when dealing with delicate electronics, like low voltage, high speed logic and very low signal amplifiers, this ground loop induced voltage drop between component reference potentials will create havoc, like rattle and hum in audio equipment or spontaneous resetting or crashing of computers.
Ground loops can also happen within the context of a single device. One of the greatest examples of this I ever saw was a huge toroid transformer in a steel enclosure with a steel bolt through it, making a loop through the center of the transformer. Actually so much current flowed through this what was essentially a short circuited extra winding on the transformer, it heated up the cabinet!

What to do against ground loops. The best is not to have them. So break them, or remove the other feature, the hole. Both work. Breaking a ground loop usually takes the form of NOT connecting a screened cable's screen at both ends, but only at the input end *. The second involves laying the sreened cable as close to the earth connection as possible. Yes, physically by running your signal cable along the same route as the power cable, if that is what carries the earth connection.
In general, to avoid earth loops getting to you, KEEP ALERT when designing stuff. Electronics is NOT easy, whatever you think of yourself. Using STAR POINTS for your ground, both inside and outside your box will get you a long way. Proper RF shunting to maintain a closed screen for RF, but not through your PCB tracks is another one. And then there is always galvanic isolation to the rescue. Components like transformers (RJ45 Ethernet will not cause you ground loops because it's all galvanicly isolated and independant from ground!), opto couplers and fiberoptic cables are choices that are all available to you, all you have to do is use them.

* There is an RF issue involved here, but this is outside the scope of my story, so I won't waste too many words on that, but in a nutshell a ground loop is almost inevitable if you want to maintain proper screening against RF, but you can guard yourself against that by using shunt capacitors at the connectors so the RF currents flowing through the screen can not "enter" your equipment.