It sounds like you have the 17 R4 and not the M17x R4 as you currently have PC4 memory. The M17x R4 has PC3 memory. 

You need DDR4 2400Mhz SODIMM memory. Make sure you get SODIMM memory for laptops. To get 32gb you will need 4 x 8gb sticks.

Crucial is the best place for Dell memory. Also check Dell and your manual for the configuration supported.

Memory RAM & SSD Upgrades | alienware | alienware notebooks | M17xR4 | Crucial.com