It's a little more complicated... You should convert from decimal to hexadecimal (or binary) and use this to make your calculation and the result you convert again to decimal (or ASCII) to put in the display...

I think that there are some routines in the code library in this site to do calculations in 16 bits format... maybe with these routines you can find enough material to develop your routines.

You really need work with floating point??? If you are trying to do a 8051 calculator I assume that it is an exercise of some kind and I think that you'll exercise enough just with integers :-)