In looking over your code fragment I do not understand your usage of R2,R4 and R6, particularly since you base your exit on whether carry(borrow) sets when you subtract R4 from R6. I would think that the R7 minus R5 operation should determine the exit point, assuming that R7 holds the number from which you wish to extract the square root. Possibly the routine below would work a little better...

mov R6,#01 ; our subtractor
mov R5,#0 ; our answer

SQLOOP:
clr C
mov A,R7
subb A,R6
jc DONE ; exit if borrow is set, we're done
mov R7,A ; save new value
inc R5 ; increment answer by one
inc R6
inc R6 ; add two to R6
sjmp SQLOOP

DONE: (final answer in R5)

If you wish to implement a routine that would allow for getting answers with decimals (such as SqRT 143 = 11.958...) using the Sum of Odd Integer method, it will require 16 or 32 bit math because the working remainders can become extremely large. The concept is that the given number is grouped off into two digit groups starting at the decimal point and working left (143 becomes 01 43). The leftmost group could be one digit. Then, starting with the leftmost group begin subtracting 1, 3, 5, etc until a borrow occurs, counting each subtraction for your answer. Restore that group to the last successful subtraction and then bring down the next group. For 143 we had one successful subtraction ( 1-1 ) for which our answer so far is 1. Now bring down the 43 (like you would in long division on paper), and multiply your answer by 10. Now you need to generate a new subtractor by taking your answer, double it and add one. Now begin subtracting your new subtractor (21) from the number you brought down (43) incrementing the subtractor by two after each successful subtraction, and again you will have one successful subtraction. Your answer is now at 11 with a remainder of 22. Now bring down two zeros so that the remainder is 2200. Again, multiply your answer by 10, keeping in mind that it is not 110 but 11.0, and double it plus one for your new subtractor (221). Now you will have 9 successful subtractions so your answer has developed to 119 which really is 11.9. This process can continue for as long as you wish, bearing in mind that the remainders and subtractors get larger and larger very quickly.
I would think the process is doable on an 8052 as long as the decimal places in the answer are kept to 2 or 3.

Hal