Dell VxRail: Some firmware not getting updated on ESXi host though the VxRail system upgrade has been completed without error.

Summary: VxRail cluster upgrade is competed without error, but some firmware upgrade is skipped causing VxRail compliance report complains the cluster is not complaint.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

After VxRail cluster upgrade is completed without error, check compliance drift report, it shows and error message about the cluster not being complaint.
In below example, it is complaining about NIC Firmware for x710 and bcm5741X on one of the ESXi hosts.

Error message about the cluster not being complaint.

Check lcm-web.log on VxRail manager, LCM first query current available firmware versions from the host to determine how many firmware must be upgraded.
The NIC bcm5741X and NIC x710 firmware are not included in the query result, so the firmware upgrade is skipped.
  
2023-09-08 17:47:35,865 INFO [LCM] [lcm-node-0] c.v.c.c.v.PlatformServiceDOClient [PlatformServiceDOClient.java:344] The firmware version on host sn xxxxxx is [LCMFirmware\{type='backplane', model='expander', version='2.52'}, LCMFirmware\{type='bmc', model='', version='5.10.30.201'}, LCMFirmware\{type='boss', model='boss-s1', version='2.5.13.3024'}, LCMFirmware\{type='disk', model='MZILS400HEGR0D3', version='DWL7'}, LCMFirmware\{type='disk', model='AL14SEB120NY', version='EA05'}, LCMFirmware\{type='disk_ctlr', model='hba330adp', version='16.17.01.00'}, LCMFirmware\{type='psu', model='CMPGM', version='00.23.32'}]

2023-09-08 17:47:35,866 INFO [LCM] [lcm-node-0] c.v.l.s.LcmNodeUtilService [LcmNodeUtilService.java:272] Did not find any matched firmware on host esxi-5 for upgrade bundle 'NIC Firmware for bcm5741X'. Skip upgrade hence.
2023-09-08 17:47:35,866 INFO [LCM] [lcm-node-0] c.d.v.l.d.p.m.UpgradeBundleProfile [UpgradeBundleProfile.java:340] Network_Firmware_8FKR1_WN64_22.31.13.70.EXE addSkipTarget esxi-5
2023-09-08 17:47:35,866 INFO [LCM] [lcm-node-0] c.v.l.s.LcmNodeUtilService [LcmNodeUtilService.java:272] Did not find any matched firmware on host esxi-5 for upgrade bundle 'NIC Firmware for x710'. Skip upgrade hence.
2023-09-08 17:47:35,866 INFO [LCM] [lcm-node-0] c.d.v.l.d.p.m.UpgradeBundleProfile [UpgradeBundleProfile.java:340] Network_Firmware_9NPPG_WN64_22.0.9_A00.EXE addSkipTarget esxi-5

Cause

iDRAC cannot stable return the host firmware inventory to VxRail, so there is a chance to this issue to happen if iDRAC did not return the firmware inventory as expected, and the relevant firmware upgrade was skipped during VxRail LCM process.

It is observed in a VCF on VxRail environment when upgrading the cluster from VCF 4.5 with VxRail 7.0.400 to VCF 5.0.0.1 with VxRail 8.0.100

But we believe this iDRAC unstable issue is not limited to VCF on VxRail environment.

Resolution

When this issue occurs, the skipped firmware must be manually upgraded to make the cluster complaint.

1. Extract the VxRail composite upgrade bundle and find the skipped firmware upgrade files. 
2. Follow KB 000134013 to manually upgrade the skipped firmware one by one via iDRAC.
Article Properties
Article Number: 000218534
Article Type: Solution
Last Modified: 17 May 2024
Version:  3
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.