Alienware Aurora Desktop AA-3795CSB (old BestBuy computer)

I just picked up an old (old as far as todays technology goes) Alienware Aurora AA-3795CSB on ebay. It just shipped today so I haven't actually seen what is inside but by the description it is still configured the way it was when BestBuy sold it. What I plan to do is set it up here at home for the grandkids (ages 13, 15, 17) to play on when they visit, and maybe try some of the (very?) slow action games myself. What I'd like to know is which video board upgrade should I use for the grandkid's newer games without affecting too much else, like GDDR3 vs GDDR5 RAM?

I was reading up a little on the GTX460 (they don't all have the same specs?) and was wondering which manufacturer or part number to buy. Also would that cause the need to change out the existing RAM, again GDDR3 vs GDDR5? I don't know what Direct 3D 11 is but people seem to think it's the thing to have so I figured the 460 would be a good bet if it will work. A second 1.8GB GTX260 is also an option I guess but then it wouldn't have the coveted Direct 3D 11.

I'm 67 and obviously I'm not up on the technical aspects of all this so keep the big words to a minimum please. :) All help is appreciated.