GSM modems are notorious for their power supply demands - both in terms of meeting their input requirements and keeping their noise from the rest of the system!

Does the manufacturer give any guidance on the requirements for filtering, etc?

It does seem odd that it only happens with the aerial on - but it could be that the RF stages can detect the mismatch of a missing antenna, and turn the power down to avoid damage?
This could be expected to reduce the "nasties" on the supply lines...