+/- 120 RPM at 4000 RPM is appalling ! I think you are working to hard, and that the correction to the motor setpoint only needs to be done 1 per rev at high speeds, and say, 4 times per rev at low speeds. Like I said the motor inertia dominates the high frequency response.

I have a motor servo running at between 4000 RPM +/- 0.1 RPM and 100 rpm +/- 0.1 RPM , with a very basic digital servo. I have a tachogenerator rather than an encoder. We update the setpoint every 0.5 seconds.

Steve