17 R1, discrete GPU not detected in BIOS

Yes you can definitely upgrade your GPU to the 980M. If it tells you that the discrete GPU is Not Detected in the BIOS it usually means that the card is dead. It could also be that the 980M has the wrong vBIOS on it and your BIOS can't read the card. It would usually still detect the card though and also give your an Code 43. 

When I got the GTX 1070 for my M18x it gave me an Code 43 and a new vBIOS sorted it out. It's possible that your 860M had its vBIOS corrupted somehow. 

Hey, i have the same problem. Also alienware 17 with same hardware (gtx860m) and it shows error 43, tried everything even reinstalling windows but nothing works, last hope is the bios thing u talked about i guess, how do i do it? Weird thing is that in device manager it still says gtx 860m but cant use it cause of the error please help

A lot of the time a Code 43 means the card is failing. 

You can try to fix it by trying an older driver.

You can also try disabling the card in Device Manager, restarting and enabling it again. I have heard this working for someone once.

If you want to try flashing a vBIOS then you'll have to download NVFlash and make an NVFlash folder in your C: drive to extract the files to. Then you'll need to search TechPowerUp for a vBIOS file and you put this file into the NVFlash folder.

Look online for guides on how to do the next steps. There are a few good guides and videos out there. 

Before flashing it's best to save your old vBIOS. You'll need GPU-Z to do this. You may find that with a driver installed, GPU-Z will not show your current vBIOS version so you will need to completely remove the driver (DDU is best) and then it should show up in GPU-Z. There is a little button next to the vBIOS name in GPU-Z that you can click to save it on your PC. 

Reinstalled the bios today but still error 43, guess the gpu is broken then but replaces are so expensive, guess this alienware is sadly going to the trash then, no dell for me anymore

Nooooooooo!!! It's a 6-7 year old laptop so it's done pretty well and it's really easy to put a new GPU in it. You can even go up to a GTX 1060 or 1070. 

Try installing a few older drivers

Already tried the driver that came with the laptop, doesnt work probably overheated or something idk, but also i dont want to invest 100€+ in an old gpu to replace it poor student here haha

Well if you don't want to buy a GPU it's still good for a general laptop and you can still get some decent money for it on ebay. There are a lot of people who still like these laptops as it was the last truly upgradeable Alienware 17" laptop. 

I love rebuilding the 17 R1's. I have two at the moment that also need GPU's.