sharing cable connection setup

I am connecting to the internet through a cable modem for my desktop. I just got a Lynksys WRT54GS router. I want to connect my laptop in the next room but NOT wireless. I have all the wires necessary for the connection and understand that the router needs to be connected to the modem and then both the desktop and laptop to connect to the router. My question is that I don't necessarily need to setup a "network". I just want to be able to give each computer internet access. I have printed out my config settings and both puters are running Win2kpro. Lynksys says to install the CD prior to hooking up the router but do I need to disconnect my modem? I know this sound simple to most but I am aprehensive about doing this since all is running well right now, any help would be appreciated.