Here's what's new !

I tried again by putting a single stick of ram in slot Dimm5, which is the second channel for Dimm1. The power light then went to solid white instead of amber, which I guess means it POSTs, but I still think it doesn't boot as I get no video.

I know the gpu works as I've tested it with another system. It's possible that the issue here is that the pc still requires a stick in Dimm1, in which case it could be an issue with the slot itself ?

I have no idea what to test next. Could it be that the 7910 doesn't support dual channel memory ? It's the only specification of my ram I'm unsure of.

What is the model number of the RAM?

Precision T7910 Owner's Manual online 

It's M393A5143DB0 - CPB.

Manufacturer is Samsung.

Thanks for the info.  I was concerned it might be KVR (Kingston Value RAM), which can be hit or miss in Dell's.

Be sure to follow the 7910 RAM Chart if not already.

About the only thing I can think of suggesting is try Crucial RAM.  It's guaranteed to work in Dell's.

You mentioned resetting CMOS, which is good troubleshooting.  Be sure to use a brand new battery.

What GPU is being used?

With the GTX 580, I take it all connections were there?

Before we try anything else for GPU's, let's try going in the order of battery, running Diagnostics if possible, RAM, then GPU.  Besides, you were able to verify the GPU(s) as working.

Are you able to access F2 BIOS or the F12 Boot and Diagnostics menu?  Try that after replacing the battery.  If it works, try running Diagnostics.  I'm hoping we're not dealing with a bricked MB.

Even though the T7910 was introduced in 2014, I attached the spec sheet below.  It has a big list of compatible GPU's.  Also, one can check out userbenchmark for GPU's currently ran in the 7910.  Just giving ideas there.

I guess I'll try to get my hand on some crucial then. I should also change the motherboard battery now you mention it.

I have followed the 7910 RAM chart up until now.

For the GPU I've tried with both a GTX580 3gb and a radeon R7 240.
(It's placeholder for a couple of Titan X's)

Also, what's being used for a boot drive?

Indeed the 580 had all of its connectors.
I'm finally testing with a Geforce Titan X (They arrived, yay!), which I tested on a different pc so I know it works.

The battery is changed, but it's no different: I cannot access anything. I do not get any sort of menu or bios options on startup. The screen gets no signal (but does kick in when I turn on the computer, so it sees it should be getting something).
I still tried plugging in a keyboard and blindly pressing F2/F12, which did nothing.

When it comes to memory I tried using one of my friend's 3200mhz DDR4 non ECC modules.
I got the white Power light instead of the blinking Amber when I put it in slot 1.

It's theoretically not compatible so I'm not surprised that still nothing happens, but if changing ram changed something, it most likely means I just need to get better Ram (like that crucial you mentioned).

I doubt it's a bricked MB as I'd most likely get a "motherboard failure" error code from the power button if it were the case.

I'm currently not using any boot drive, I'm just trying to reach the bios for now.

I take it the monitor too is from a working PC.

I'm thinking in spite of RAM, it should let you access BIOS or Boot Menu.  We'll have to see what it does if you get the Crucial RAM.  By any chance did you try it without any RAM to see what it does?  It should still give the RAM error code, but I'm curious if anything changes there.