"it is possible to find a set of table values that permit the high voltage to settle into a reasonably stable voltage as long as the load on the HV output stays relatively constant. "

If you have come to this point, then most of the hurdles have been crossed. It is just that while on load, your control software correction rate and the response of the regulating loop dont match.

Could be due to ,

A. On load, the voltage correction slew rate is not adequate as compared to no-load.

B. The HV source is unable to meet the load demand and hence is unable to rise.

One way to come close to observe the phenomenon is to reach the stable level in no load mode, then "manually" try and vary the load SLOWLY and observe the error level vis-a-vis the correction command. And keep on increasing the frequency of variation till the correction loop breaks.

OK for the above - the load MUST be variable manually.

Raghu