Phil,

the easiest way is to connect the cathode of the LED to the desired port pin and the anode with a resistor to +5V. The value of the resistor is about:
R = (VCC-ULED) / ILED = (5V - 2V) / 2mA = 1,5 kOhm
This gives you:
port pin = 0: LED on
port pin = 1: LED off

If you want your LEDs on with port pin = 1 you need any kind of driver (something that can source some miliamps into the LEd if switched on). Choose some IC drivers or - if you prefer the "discrete solution" - some NPN bipolar transistors or N-channel enhancement MOSFETs. For the bipolars connect the transistor's base with 20kOhm to the port pin, the emitter to GND and the collector with 1,5 kOhm and the LED to +5V. Et voilà.