"insufficient system(PC) memory??? How is that possible?"

Very easily!
Have you never seen an 'Insufficient Memory' or 'Insufficient System Resources' PC error message before??

"If the problem is with the PC memory, then KEIL should not be able to compile generate the hex file???"

Not necessarily!
Maybe SDCC is just grossly inefficient in its use of PC memory resources?

Anyhow, if you're sure that it's not a PC resource problem, what about the other options?
What does "DRAM" mean in this context?
What does the Manual say??