Latest Server Update Utility (SUU) onboard NIC firmware flash fails. NICs no longer detected. Poweredge Server not booting.

Hello RoyBuckley,

I'm sorry to see the NIC update did not complete successfully. 

What is your system model? BIOS and DRAC firmware version?

Try a flea power drain, NVRAM clear,  then see if you can get in the LifeCycle controller to rollback the update:

drain flea power (shut down, disconnect power cables and NIC cables, leave it off 5 minutes)

With it off move the NVRAM jumper to the clear position.

There is a map under the lid.

Then reconnect power

Power up to the NVRAM error message, shut down and return jumper to default position.

LifeCycle firmware rollback : https://dell.to/3hhE7ZD

Please let me know how it goes.

Is there anyway to recover from a bad onboard intel NIC firmware update?

Hi Charles,

These steps (flea power drain, NVRAM clear) were tried when the problem happened but no luck unfortunately.

We can flash firmware for bios and idrac via idrac lifecycle manager uploading the files from a remote PC but it won't take any firmware for the NIC. Tried the lastest BIOS and iDrac. We also downgraded both and upgraded again. NIC however refuses to take any firmware old or new.

I suspect that since the network flash failed half way, the NIC is bricked. Once the nic is bricked the system board does not POST

Do you know any secret way to recover bricked nic firmware from a bad flash or any way to remove a bricked on-board nic from the board to allow progress past POST?

Hello RoyBuckley,

I am curious if you are able to enter F10 Lifecycle controller? You indicated you could not access any F functions on POST. Is that still true?
This also makes me wonder how you were able to upgrade and downgrade BIOS as the system must enter the LifeCycle controller to complete a BIOS update. 

What is your system model?

Try to reset the LCC like this and then try the rollback:
-boot to F2 -> System Setup -> iDRAC Settings -> Lifecycle Controller to check status
-Choose: Lifecycle Controller set to Enabled 
-Collect System Inventory on Restart set to Enabled. 
-Cancel Lifecycle Controller Actions set to Yes.
-save, exit, reboot.

LifeCycle firmware rollback : https://dell.to/3jOUZrc

Hi Charles,

After the dell logo etc, it hangs at the press F2, F10, F11 screen. Any key that you press, the option highlights but it's stuck and doesn't proceed past this point. It is a Dell T630 out of warranty.

Try to reset the LCC like this and then try the rollback:
-boot to F2 -> System Setup -> iDRAC Settings -> Lifecycle Controller to check status
-Choose: Lifecycle Controller set to Enabled 
-Collect System Inventory on Restart set to Enabled. 
-Cancel Lifecycle Controller Actions set to Yes.
-save, exit, reboot.

Thanks for the step. F2 does nothing unfortunately. 

You can't load lifecycle from any Fx key. However iDrac connection from a PC to the server works fine. That is what I am using to interact with the server. The firmware is uploaded from the PC and scheduled to install immediately. The server reboots and the lifecycle manager loads up on screen, does the firmware update and reboots back to the stuck F2, F10, F11 screen.

I've upgraded and downgraded both BIOS and iDrac/Lifecycle firmware without any issues.

However NIC firmware upgrade/downgrade attempts all fail and the lifecycle controller logs from iDrac say the NIC firmware upgrade failed and onboard NICs not detected (bricked NIC)

Something about bricked onboard NIC causes it to freeze at the F2, F10, F11 screen.

Hi Charles,

Thank you.

I've tried minimum components boot and firmware roll back but without any success unfortunately.

The system boards are not cheap but it looks like that is the next step. I will think about it.

Thank you again.

Hello RoyBuckley,

One final ditch effort, try our Bootable ISO for the T630 to see if the firmware will take : https://dell.to/33cVZiz;

Let me provide some motherboard part numbers in case you need it:

PN#W9WXC - Motherboard ASSY,PWA,PLNR,SV,STPM,PET630 
PN#NT78X   - Motherboard ASSY,PWA,PLNR,SV,STPM,PET630,V2

it wont boot from the optical drive unfortunately.

Thank you kindly for the part numbers.

I've patched 100's of servers with the SUU windows app and this is the first time a failure has happened.