Ah! How delicious :o) I haven't seen this one for a while.

This is actually a well known phenomenon in CMOS circuits:

A CMOS input is diode clamped to both GND and Vcc to prevent damage from ESD.
What happens is that any output of the USB device that is 'high' will supply the rest of your board through the Vcc clamping diode on the input of the micro. Of course the output probably cannot supply enough current and there's a resistor in series anyway, so that is why you measure 3V.

To prevent this, you can place a Schottky diode in series with the cathode connected to the output and add a pull-up resistor to the input. This only works if the pins are unidirectional, of course.

Best regards,
Rob.