1) The modems you have are these dial-up modems?
2) is the modem on the PC-side an external or internal modem?
3) In case you have dial-up modems, can you send your modem AT-commands manually at all the speeds you mention?
4) Same as q3 but for your 8051 modem.

can you test your modems as such:

PC --- Modem --- Modem --- PC

eventually using only one PC and two serial ports.

If you say a connection cannot be established, what exactly do you mean? Do you here your modem dial but the rmeote site does not answer? Or doesn't your modem dial at all? Or is it the handshaking that goes wrong?

regards
Patrick