Hi!
As far I understand,
*, You system is working.
*, You are afraid of lot of heat generated in you system.

What I suggest,
1, Don't worry about the heat, it will not harm you system if you can touch it with out pain.
2, The configuration of power supply you have told us (12-0-12VAC may be bridge rectified then 7805) is used for places where you expect AC main variation. So If this is the case you can only add more and more heatsink until its heat is distributed enough so that you can easily touch. OR use switching power supply.
3, All the heat is generated I think from in 7805. Two parameters can reduce it, i,Reduce input voltage, ii,Reduce current consumption in your system.

By the way I have design a system in with 89C2051 is used to drive 35A relay through transistor buffer and also have problem of AC mains variation. So I design a high input voltage 7805 regulator just like you and it go heated also but it never harm system even a continous operation of 2 months as far as i know.