If you want to make your own modem
You could get a chipset from Rockwell or somebody and try to control it directly from your 8052 but you would need to follow the telephone line interface reference design exactly.All telephone line interfaces are tested and approved by the telecomms regulators to ensure that you are not going to kill the exchange,and its not going to catch fire if the line is hit by lightening.