VxRail: After Host Remediation is Completed, Compliance Check Reports the Host as "non-compliant" and the Image on the Host is Mismatched
Summary: Unable to proceed with vLCM upgrade on the cluster due to incompatible third-party VIBs.
Symptoms
With vLCM enabled, performing an upgrade using VxRail UI fails because the third-party drivers are not part of the baseline image used by vLCM.
2022-08-24 14:32:17,926 WARN [LCM] [lcm-core-0] c.e.m.m.u.s.VirtualApplianceUpgradeService [VirtualApplianceUpgradeService.java:1114] LCM Exception occurred during upgrade for virtual appliance 70372188 - {}
com.vce.lcm.exception.LCMException: Failure occurred while running an upgrade for bundle: VXRAIL_COMPOSITE_7.0.372-27602687_Installer_for_7.0.x.zip. The error message: 1 error(s) meets. Details: 1.After host 'zzzzzzzzzzz.local' remediation completed, compliance check reported host as 'non-compliant'. The image on the host does not match the image set for the cluster. Retry the cluster remediation operation..Resolutions: [host-2040]: No resolution for the provided vCenter...
at com.vce.lcm.api.LCMServiceImpl.handleUpgradeException(LCMServiceImpl.java:1745)
at com.vce.lcm.api.LCMServiceImpl.resumeUpgradeAfterReboot(LCMServiceImpl.java:644)
at com.emc.mystic.manager.upgrade.service.VirtualApplianceUpgradeService.resumeUpgradeAfterReboot(VirtualApplianceUpgradeService.java:1103)
at com.emc.mystic.manager.upgrade.service.VirtualApplianceUpgradeService$3.run(VirtualApplianceUpgradeService.java:901)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: com.vce.lcm.exception.LCMException: 1 error(s) meets. Details: 1.After host 'zzzzzzzzzz.local' remediation completed, compliance check reported host as 'non-compliant'. The image on the host does not match the image set for the cluster. Retry the cluster remediation operation..Resolutions: [host-2040]: No resolution for the provided vCenter...
Cause
The third-party drivers installed are not compatible, or the OEM .zip version of the driver is not uploaded to vLCM.
See the Additional Info section of this article for how to add an OEM .zip to vLCM.
Resolution
If vLCM is already enabled, and unsupported third-party drivers are present (no OEM .zip), contact Dell Support.
During enabling vLCM, the scan ignores customized components and sets the wanted state as NONE for those installed customized components. During vLCM enablement, remediation is triggered automatically, and the customized components are removed.
If an OEM .zip version of the driver exists, see the Additional Info section of this article for how to add a OEM .zip to vLCM.
This issue is addressed in the VxRail 7.0.400 release.
For VxRail releases that are earlier than 7.0.400:
- If the enabling vLCM is planned, the VxRail cluster should be upgraded to 7.0.400 or later first, then enabling vLCM.
Additional Information
vLCM gives the option to upload third-party drivers (VIBs) as part of the upgrade process. The correct process to add these drivers is as follows.
- vLCM is already enabled
- Upload the composite bundle
- Click the CUSTOMIZE button
- Click ADD COMPONENT
- Browse and select OEM ZIP add-on
In the failed upgrade example above, there already was a third-party driver installed. This was not added to the OEM ZIP add-on after composite bundle was uploaded, and the upgrade was started.
In this scenario, as the third-party drivers were not part of the image used by vLCM. vLCM attempts to remove the drivers (by design), and during VxRail compliance scan, it fails the upgrade.
If vLCM is chosen to be used, all OEM ZIP add-ons for the drivers must be available and provided by the vendor. The vendor must confirm vLCM compatibility.
See VMware article Bulletins, Components, Add-ons, and ESXi Base Images .