The reason your program does not run under the Keil simulator is because after you send data out the serial port, the program continues executing whatever data it finds. The Keil simulator alerts you to this problem by indicating that there was an Access Violation and that there was no read or execute permission at the address after the CLR TI.

You need a JMP after writing the transmit character to SBUF.

On a real devices, this MAY work because the PC eventually wraps around to 0 and starts executing from the beginning of the program. However, this is certainly not what you designed and it is certainly a bug.

Another problem I found does not technically make the program FAIL, however, it certainly makes it run more slowly. In communications programs it is customary to check the status of the transmit buffer BEFORE you setup to send a character. After you setup the transmit, your program can go on running. It only has to delay the next time you need to send a character (if the previous character is not finished transmitting). If you check the transmit status AFTER you setup to send a character, your whole program is always delayed for one character time.

Following is your program with the changes I made. It works fine with the Keil simulator.

ORG 0000H 
DYTA EQU 50H 
AJMP START 


START: MOV TMOD, #20H ;set timer1 to mode 2 
MOV SCON, #01010000B ;set UART to mode 1 
MOV TH1, #0FDH ;Fosc=12M, baudrate=9600 
MOV TL1, #0FDH 
SETB TR1 

; Set the TI bit since you can start transmitting
; when the program starts (there is nothing in
; the transmit buffer)
SETB TI

LOOP:
JNB RI, $ 
MOV A, SBUF 
MOV DYTA, A 
CLR RI 


; Move TI test above sending a character
; This lets the program run while transmitting
; Otherwise, the program does not get transmit
; at the full baud rate
JNB TI, $ 
MOV SBUF, DYTA 
CLR TI 

; Add a JMP so that the program
; Jumps back to get another character
JMP LOOP

END 

You may have noticed that most assembly programs have a type of block indenting that makes it easier to read instructions and assembler directives. Doing something like that yourself might make your programs easier for others to read and comment on.

Jon