Like Michael already mentioned, your application heavily suffers from neglecting a solid ground plane!!
And, like I already stated in my last reply, the connectors on your PCB are placed in the wrong way. Both mistakes cumulate: In combination with neglecting the use of a solid ground plane you result in total disaster!

Why?
Assume interference hits your system somehow, then a current can flow through your entire system. If all connectors sit at only one edge of your PCB and if a solid ground plane is used, then the interference stops at the edge where the connectors sit and does not furtherly flow across your PCB.
But with your design any interference running from one connector to some other will always run over your PCB and force very strong voltage drops across your very thin and narrow ground copper traces. It's the inductivity of these narrow copper traces which is responsible for the develop of enormeous ground noise, enough to totally ruin your signal integrity.

You told, that you use an analog ground and a digital ground. That's ok, but you must not route these both grounds entirely wild over the whole PCB. You must divide your PCB into an analog and digital section, and must avoid all intermixing of both grounds. They have to be very accurately kept separated! A connection of both grounds is only allowed at one single point, best near the micro.

I'm sorry, but your problems cannot be solved by any little repairs. You must totally redesign your PCB.

Good luck,
Kai