I am using LM324 for the design. The initial calculation puts me on a 92mV reference. I was wondering if the noise might throw this reference out of the window if I use a simple voltage divider

The noise on the reference can easily be handled by proper decouplin, however I am 98.17% sure that the noise on the ADC signal input is more than 92mV/resolution. So without amplifying the signal to "make the ADC happy" you will get a very noisy result.

Most ADcs I have seen have a noise specified as +- n bits at full range. So if your ADC is 10 bits and has a noise of +- 1 bit at 5V that convert to 5mV which is a 5% error at 100 mV full scale.

Erik