Christian,

There are different types of optical encoders. I suppose that you refer to a dynamic (not absolute) rotary encoder. This device usually gives you two output signals with a 90º phase shift between them.
You can connect one output to P3.2 (using external interrupt 0, edge-triggered) and write a simple ISR. The ISR has to read the other output state (connected to any port, say P1.0). That state indicates the direction, left or right.
So, the routine can be very simple, something like this:

unsigned char direction; // global

// Function name and syntax depends on the compiler you use.
void interrupt EXT0_int()
{
if(P1.0) direction= 1; // maybe left
else direction= 2; // maybe right
}

and your main program can use 'direction' this way:

...
if(direction){
if(direction==1){...}
else{...}
direction= 0; // clears the 'command'
}
...

If you need a fast response to the encoder, move the code from the main program to the ISR. For example you can increment or decrement a SW counter within the ISR in response to the encoder signals (left or right).

Regards,

Alfredo del Río.