XPS 13 9360 red grub boot menu

I tried different versions of Ubuntu to see what works and what doesn't. The last time I installed Ubuntu I specified 'nvme0' as the drive to boot form, instead of 'nvme0n1'. That was a mistake. When it rebooted after the installation was finished I found myself staring at a red grub boot menu telling me:

GNU GRUB  version 2.02~beta2-36ubuntu3

*If you are seeing this menu, your installation has failed.
 Choose from the following options for debugging purposes:

 Single user mode
 Automated installation of Ubuntu 16.04 (Default)

When choosing 'Automated installation' it sort of re-installs, without giving me any option of configuring the installation. It ends up booting in the same red grub boot menu.

Choosing 'Single user mode' makes it do some stuff and then I see:

Welcome to rescue mode! After logging in, type...
Press Enter for maintenance
(or press Control-D to continue): _

I press Control-D and a version of Ubuntu launches.

I also tried inserting the dell recovery USB-stick I made and then launching it via the bios boot menu. But again I go straight to the red grub boot menu.

Logically thinking I'd say grub need to be removed from the nvme0 disk. I have no clue how to do that though. I don't even know what it is actually. You'd say 'nvme0n1p1' through 4 are the partitions and 'nvme0n1' is the disk. But what then is 'nvme0'?

Anyone an idea what I could try?