"the problem is that the system works fine under 15000 rpm but when rpm exceeds 15000 then starts behaving abnormal."

define "abnormal".

how do you measure? how long the sampling time? for each rotation from the motor, how many pulses are driven through the optocoupler? how are you counting pulses? 8 bit or 16 bit counter?

too long sampling time = too many pulses = counter overflow.

sorry asking too many questions, but the crystal ball went to polishing :-)