Your connection is most probably not correct, since it doesn't work for you :)

What kind of 8051 system do you have? The 8051 chip can not be connected directly to the PC. You need a level converter (such as MAX232) to convert between the digital levels of the 8051 chip to the levels used on a RS232 serial cable.

THe UART in the PC must be OK, if you can use it with your other PC.

The pinout of the serial port on the PC is very well documented.
If you have a development board with the 8051, then it probably uses the same pinout. If not, it should be supplied with documentation telling what connectors the development board has, and where the signals are located on the connectors.
If you do not have a development board with an existing RS232 connector, then you have to look at the datasheets for the specific chips to figure out which chip pins that have the required signals.

Yes, you can use a scope or a DMM to check levels. But before you do, start with making sure that you have two RS232 devices (i.e. that the 8051 side has a level converter). Then just make sure that you map the RX, TX and GND signals correctly. With a DMM, the 8051 will have 0 or 5V on the signals (if it is a processor powered by 5V). If the 8051 board has a RS232 port, you should have positive or negative voltages. Normally maybe +3 .. +15V or -3 .. -15V. Same thing for the output signals on the PC. With a scope, you will be able to see not just the voltages, but also if any data is sent out, you can see the baudrate and how many bits that are sent for a single character.

Another thing: The PC may require that you turn off hardware flow-control, to be able to send data to the 8051. And you must make sure that you have correct baudrate and format (number of bits, parity, stop bits, ...) configured on both sides.

In your post, you only say "there is no reply (There must be a login message at reset)". Does the 8051 have a program that shows a boot message when reset? How do you know? A scope will show if any data is sent out when you reset the processor. And you can follow this signal all the way to the receive pin on the PC.