That setup should be fine - are you sure that both PC & 89C51 have exactly the same settings?

Rather than try to debug both your embedded software and your VB at the same time, it's better to start by just using a simple terminal emulator on the PC to verify that the data is arriving correctly on the COM port; once you're sure of that, then you can start trying to get your VB working!

BTW: try a search for "VB" and "MSCOMM" - they've come up before