check the line from the wall, usually this goes to your DSL modem.

all you need to do is connect the line that passes from your DSL modem to your computers ethernet port.

 

When you are connected via your router, the source DSL line (which is like a phone line) goes into your DSL modem, an ethernet cable (let's call this cable A) connects the DSL modem to the router, and another ethernet cable (let's call this cable B) goes from the router to your PC's ethernet port.  And I'm assuming this is all working fine when connected as described. Without the router, just unplug cable A at the DSL modem (you don't need this cable anymore), unplug cable B at the router and plug it into the DSL modem. If you still cannot access the Internet, make sure that you are logged onto your ISP. Perhaps this log-on service was provided by the router which has been taken out of the picture. Go through the standard Window's steps to set up an Internet connection: Control Panel, Network Connections, Create a New Connection, and just follow the prompts.