PowerEdge R730 + Broadcom 57800S Upgrade - PCIe Link Training Failure

Question

Hi all. I recently purchased a PowerEdge R730 server for use in my first homelab setup. The server came with the quad 1 gigabit NDC (Broadcom 5720 Quad Port 1GbE). I'm attempting to upgrade to the Broadcom 57800S card (with dual 1 gigabit RJ45 and dual 10 gigabit SFP+ ports).

I powered off the server, disconnected power and swapped in the new NDC (being very careful to align it properly, applying gentle pressure to seat the connector and starting the threads on the screws to install it by hand). When I tried to start the server, I got the following error message: "UEFI0067: A PCIe link training failure is observed in Embedded Network Device and the link is disabled. Do one of the following: 1) Turn off the input power to the system and turn on again. 2) Update the PCIe device firmware. If the issue persists, contact your service provider."

I tried rebooting the server, and got the same result. I tried shutting it down, disconnecting power and reseating the card, and got the same result. I then tried to update the firmware through iDRAC (using the "Dell firmware DUP for Qlogic BCM57xxx series adapters", Version: 15.05.12 ,A00-00, File Name: Network_Firmware_T3KH2_WN64_15.05.12_A00-00.EXE), but when I uploaded the file to the server, I got the following error: "Unable to complete the firmware update operation because the specified firmware image is for a component that is not in the target system inventory."

I should also state the quad 1 gigabit card I had in there previously was working properly.

I'd appreciate if anyone could offer more information about this problem, please, or provide any suggestions for further testing.

I've included screenshots of the errors below, and a couple of the card itself (please forgive the poor quality of the pictures):

PCIe link training failure: https://i.imgur.com/4MYvjWz.jpg

iDRAC firmware update failure: https://i.imgur.com/NoUbhQh.png

NDC card: https://i.imgur.com/kdfYjC1.jpg & https://i.imgur.com/hdpGdVF.jpg (Please note that the damage to the screwhead was there when I got the board.)

Thanks for any help or advice you can give me.

Dell-DylanJ · Answer

For now, I would shut the server off, unplug it, hold the power button down for 30 seconds, then reinstall the old card. Once the old card has been reinstalled, power the server back up. You can try flashing the iDRAC again at this point, but you might also want to check your BIOS revision, too. I'd try the new card again at this point and see if you get any behavioral changes.

amp888 · Answer

Thanks for your reply. I've reinstalled the old card (the 5720), and it's working properly, as it was before. I tried to perform the NDC firmware update, but I get the same error message as above ("Unable to complete the firmware update operation because the specified firmware image is for a component that is not in the target system inventory"). It appears as though the firmware for the family that contains the 57800S doesn't contain the 5720, so they have different firmware versions? The description for the "Dell firmware DUP for Qlogic BCM57xxx series adapters" update says:

"This software bundle includes the Dell Update Package to install QLogic Firmware on the following adapters QLogic 57810S Dual 10GE PCIe Standup SFP+/DA CNA QLogic 57810S Dual 10GE PCIe Standup Base-T CNA QLogic 57800S Quad Port 2, 1GB x 2, 10Gb rNDC Base-T QLogic 57800S Quad Port 2, 1GB x 2, 10Gb rNDC SFP+/DA QLogic 57810S-K Dual Port 10Gb bNDC KR CNA QLogic 57810S-k Dual Port 10Gb bMezz KR CNA QLogic 57840S-K Quad Port 10Gb bNDC KR CNA QLogic 57840S Quad Port 10GB rNDC SFP+/DA"

I updated the system BIOS to the current version (2.11.0) just after I got the server.

Dell-DylanJ · Answer

The NDC firmware does look to be different, but that would make sense. Knowing that the BIOS is up tod ate is good, I'd still recommend flashing the iDRAC, though. It may not fix the error, but the BIOS and iDRAC would be the most sensible firmware to update.

If the card doesn't work after updating the iDRAC, I'd treat the card as bad. You know that the slot works after all, because the 5720 works fine.

PowerEdge Hardware General

PowerEdge R730 + Broadcom 57800S Upgrade - PCIe Link Training Failure

Was this post helpful?