Your router will have a IP address which is configured in each of your PC's as the GATEWAY ADDRESS. Your local LAN should be in the SAME network as the router.

i.e assuming a subnetmask of 255.255.255.0 in the router and PC's

if the router was 192.168.2.1 then the PC's should be in the range 192.168.2.2 to 192.168.2.254 i.e in the same network.

If you do this then any IP address request that is not in the local lan is passed to the router automatically.

If you are using this for Internet access then you may also need to configure DNS (Domain Name SERVER)
addresses in each of your PC's this you can get from your provider.

If you want any more info then drop me an e-mail.