So we know you can't prove a design to be correct. You can't prove a program to be correct. Testing proves nothing. Yet you must test to remove errors tests would find.
The question is: How much?
The answer is: Until it meets (and safely exceeds) the specification tollerances. Including tollerance for probability of exceeding the tollerances. (you can't guarantee EVERY device you make will be fine. You can only guarantee at most one in x will fail, and that you will replace it at no charge, plus eventually the chance it will fail in a harmful manner is one in y, and you will pay compensations in case it fails like that.)

You can't prove the specification is correct, for the simple reason that specification is arbitrary. The usual specification, which you oppose so strongly is "It just works for me" - and in this case testing that it indeed works for me is perfectly sufficient. It doesn't mean the design isn't flawed. It doesn't mean there's no errors. But nothing means that, so why bother? You make the devices to perform certain tasks, not to make them showcases of perfection. If they do the task, it's good. If they don't, it's not good. If they do the task partially, though, or not always, you must answer how often, or how much of the task do you agree to be done wrong.
If you aim at perfection, not only you will never reach it, you will waste a lot of time and resources too.