If you use a larger diameter, 3.g. the .080" that you mention, you'll regret it. The "standard" hole for a 0.025" square pin, which is what those floppy headers to which you refer use, is 0.041" and that, managed well in a high quality fixture with a very steady machine, e.g. a vertical milling machine, will typically drill .003" larger than the bit. .030" holes are generous for standard DIP sockets. If you hand-drill your holes, you can use a piece of perf-board to help guide your bit. You'll have to replace it from time to time, because your hand motion will enlarge the guide hole. I'd suggest you use a .030" bit to drill the hole and subsequently ream them with a 0.040" bit in a pin-vise using your fingers to rotate the bit. That will ensure they're not too terribly out of round. They will, of course, be oversized, but you can work out the drill sizes once you gain a bit of experience.

If you use a conventional drill motor with a conventional chuck, you'll need about one bit for every four or five holes, as you're going to break a lot of bits. If you use a Dremel tool, you'll break fewer bits, but the holes will be ovals or worse.

The holes in PCB with inserted screw-machine-manufactured socket pins are typically drilled with .050" bits, which drill somewhat oversize, typically .003" in production. When the .055" barrels of those pins are forced into the hole, they stay put, thanks to the interference fit.

RE