R740 - Fail to post, half power/ fake boot sequence?

Question

I was doing some fall cleaning and powered down the server to blow dust out of the chassis, I do this semi regularly and have never had an issue. I removed the power leads, pressed the power on button a few times and let it sit for a few minutes after pulling it out of the rack. I then used a plastic nozzle and a blower to blow dust out of the chassis. I put everything back together and pushed the server back into the rack, there was some weird resistance that seemed to be from the right rack ear area but I didnt see anything, could just feel it while pushing the server back into its seated position (so maybe something happened to the right rack ear? idk)

When I went to boot it back up, the green power led on the power button didnt light up, but the server started its usual fan ramping, disks booted up and the power and activity leds were lit up and blinking. After 30-45 seconds, the server ramps the fans down and just stays in that state. idrac reports the system is off, even though it went through the weird half powered / fake boot sequence and is now idling with disks on and fans in a low power state. I can shut the system down by holding the power button for 3-5 seconds, so it seems to me that the power button works, but the led doesnt light up. I get the same result when using the idrac gui to power on the system when it is off, and when the system gets to its failed boot state, idrac still shows the system as powered off and only shows an option to power on the server. Trying to power on the server when its in the failed state does nothing physically, but the idrac gui shows a success message.

I dont see anything of use in the lifecycle logs or anywhere else. One thing to note is that the system inventory data is wrong in the idrac gui. It still shows as all my PCIe cards are installed, all my RAM and both CPUs when I've only got 1 CPU and 1 DIMM in with 0 PCIe cards.

Steps I've taken to troubleshoot and get it down to minimum viable boot:

- removed all DIMMs, removed both CPUs and trying 1 at a time, removed all PCIe cards and disks.

- Tested the DIMMs and CPUs in another system (r540 and all the components work in that system)

- removed, cleaned and re-seated the right rack ear cables (seen a reddit post that had similar symptoms as me and they had damage to their right rack ear, replacing with an ebay part solved their issue, I have one in the mail)

When there are no CPUs installed it doesnt do the weird, fake, half booted thing, it just stays dead and there are 2 leds flashing on the mobo. I also see in the idrac gui that the system seees there are no CPUs or a CPU is removed, so the mobo and idrac are talking (but the idrac system inventory data is stale).

I have a replacement right rack ear coming to see if that will solve the issue. If it doesnt, the only thing thats left in my mind is that something happened to the mobo and I will need to get a replacement. Which is crazy, but I digress.

I'm here wondering if anyone else has run into this weird half boot/post and the power button LED not lighting up, but the power button works to turn it on and to shut it off (hold for 5 seconds).

TIA!

DELL-Charles R · Answer

Hello,

Try Clear NVRAM if you haven't already - there is a map under the lid to show the jumper location. Shut down, Set jumper to the Clear position, power on and Post up to the jumper error, shut down and return jumper to original location.

Review your Minimum to post

The minimum components to allow the PowerEdge R740 to complete POST are:

● One processor (CPU) in socket processor 1

● One memory module (DIMM) in socket A1

● One power supply unit

● Right control panel and cable (for power button functionality)

● System board

Remove anything not on that list: DVD, Hard drives, PERC controller, backplane, network card, NIC cable, any pcie cards, keyboard, mouse, USB devices, …. anything not on the list remove.

If it does not post then the issue is with one of those minimum components.

If you get successful POST, put things back a little at a time until you find the faulting component.

As you have had the system open did you try reseating all cables and components?

The ear is a good place to look in case the cable got pinched.

DELL-Rey G · Answer

The power button can get jammed into the off position, please confirm thats not the case here.Rey#Iwork4Dell

baudneo · Answer

I'm back at home later today and can try the nvram reset. I did remove all components (fans, NIC daughterboard, all risers/PCIe cards, all drives, h730p mono, backplane, etc) besides 1 cpu and 1 dimm, and still had the same behavior.

I did remove and reseat all cables. I did check the power button. It isn't sticking or anything from what I could tell. It still worked to attempt to power on (the LED didn't light up though) and when I hold the power button down while the system is "semi powered on" for 5 seconds, it shuts everything down. So from what I can tell, the power button seems physically functional.

When I get home I'll try the nvram reset. I am waiting for an eBay right rack ear to arrive to try that. If all else fails I've found a motherboard (1kpx8) for a decent price.

Follow up question: are there any gotchas for r740 motherboards? I see several different part numbers but can't find the differences. They look physically identical, so assuming different versions/variants?

baudneo · Answer

Unfortunately, the nvram reset did not change the boot behavior. I will update this post when the right rack ear arrives.

DELL-Joey C · Answer

Hi,

In your starting post, you mentioned that you had some resistance on the right side, from pushing the server back into the rack, that could involve the right control panel cabling. Maybe it got damaged from that. But you did mention the power button seems to be working from turning it on, turning it off by holding the power button. You have access to iDRAC, does the logs show any errors about the power supply? Have you tried iDRAC soft reset? https://www.dell.com/support/kbdoc/en-my/000126703/how-to-reset-the-internal-dell-remote-access-controller-idrac-on-a-poweredge-server

The R740 mainboard has different update versions, but they do not have any restrictions of using.

baudneo · Answer

Idrac and lifecycle logs don't have any logs about power or psu. There are no log entries that are errors or warnings. I haven't tried the idrac soft reset yet.

I have gotten it down to 1 cpu, 1x 1500watt psu, both left and right rack ears and 1 dimm. No fans, risers, NIC daughter/mezzanine board, extras.

The right rack ear replacement (with cables) from eBay should hopefully be here in a week so I can test that. If that doesn't fix it, I've already been shopping around for motherboards. A used r740 8xsff barebones with 2x PSU and motherboard is roughly the same price as a new/used r740 motherboard, so I may end up buying a used barebones chassis instead.

I've had to take the 16x SFF backplane and cage from my r740 and swap it with the r540 8x LFF backplane/cage in the interim to keep the homelab going, I didn't know that was possible, but it is and works without modifications. The 8x LFF cage/backplane being installed onto the r740 requires 2x cable modifications (need to lengthen the backplane signal and power cables by 1-3 inches) and also puts the SAS cables in a weird position. These things are good to know though!

Rack Servers

R740 - Fail to post, half power/ fake boot sequence?

Was this post helpful?