When you apply +5 volts to a chip like this on an input and the GND is connected but the VCC is open then the input protection diode conducts current toward the VCC pin and other I/O pins. The forward drop of the protection diode in this configuration is about 1 volt. This is what accounts for your seeing the other pins of the part at 4 volts.

It is not usually advisable to have active voltages on IC pins while the chip is not powered via its normal VCC/VDD and GND pins.[Note that there are exceptions to this rule however and sometimes it is OK to have an input pin to an IC powered even though the rest of the IC is not powered].

There is a strong likelyhood that you may have destroyed your EPROM by connecting it this way.