VxRail: Best Practices for vLCM in VxRail

Method for VxRail:

Correct Usage

• If third‑party drivers or solutions are already installed on nodes before enabling vLCM, always use the vLCM Enablement Wizard to add any standalone components.
• Never manually add components using third-party methods, as it causes upgrade failures.
• Once enabled, the vLCM depot remains persistent and does not need readding unless upgrading a component.
• Components must be added by the UI during enablement to become part of the image used for VxRail nodes.
• Components may also be added by using ADD COMPONENT feature in VxRail Manager UI (Only accessible after the composite bundle is uploaded)
• If the customer has followed Broadcom Article ID: 390098 to add Broadcom authenticated downloads or a token to the vCenter Lifecycle Manager instance that manages VxRail, please ensure that these depots are disabled prior to performing any VxRail upgrade activities. VxRail does not require access to these depots, and any connectivity issues with them can cause VxRail upgrades to fail. All required depots for VxRail come from the Upgrade Composite Bundle.

Incorrect Usage

• Avoid manually adding depots or offline bundles for third-party solutions, such as NSX, by any UI or CLI method.
• Manually installed VIBs will not be reflected in the wanted state and will be removed during the next upgrade cycle.
• Do not attempt to add Solution components by the ADD COMPONENTS feature in the VxRail plugin (blocked in code from 8.0.320 onwards).
• Do not reference any Broadcom Knowledge Base articles for VMware Lifecycle Manager when troubleshooting VxRail LCM issues, as they are not applicable.

Component Upgrades

• When upgrading a component, add the new offline bundle by ADD COMPONENTS feature during the upgrade process.
• vLCM will automatically recognize and replace the previous version with the updated component.
• VxRail Manager automates life cycle management, and any manual modifications in VMware vCenter Lifecycle Manager UI are unsupported.
• Important: Changes in VMware vCenter Lifecycle Manager UI are not reflected in VxRail Manager and causes inconsistencies and upgrade failures

Third-Party Solution Components and VxRail vLCM

Understanding Component Types

• Solution Components: Includes NSX, Rubrik, NetApp, Veeam among many others. These are managed by their own life cycle tools and are not handled by VxRail vLCM.
• Standalone Components: Includes GPU drivers, HBA drivers, PowerFlex These can be added to VxRail vLCM using by ADD COMPONENTS (depots or offline bundles - not standalone VIBs).

NSX Example

• NSX requires a complete solution upgrade (Manager, Controllers, Edges, then VIBs).
• NSX upgrades are managed independently by NSX Lifecycle Manager, and VxRail vLCM does not interact with it.
• Adding NSX depots or offline bundles anywhere outside NSX Manager breaks the life cycle state.

vLCM Configuration Notes for ROBO Clusters

• When vLCM (vSphere Lifecycle Manager) is enabled on a ROBO cluster, the witness node must also use a single image.
• Baseline-based upgrades are not supported in this configuration.
• Manual upgrades of a witness node used by a ROBO cluster in vLCM mode are unsupported.

Witness Node Sharing Guidelines

• Witness and ROBO clusters can only be shared among other ROBO clusters.
  They cannot be shared across a larger Stretched Cluster configuration.

• Shared witness nodes should only be used with VxRail clusters.
  They are not supported with VSRN clusters.

• All clusters sharing a witness node must remain within one code revision.
  If one cluster upgrades two or more revisions ahead, it may impact the stability or compatibility of other clusters sharing the same witness.

Do’s and Do nots:

Do’s

✅ Ensure lcmProperties.upgrade.advanced.mode.on=false as the default setting for vLCM.

✅ Use only depots when adding third-party components like GPU or HBA drivers.

✅ Verify compatibility of third-party components with Single Image Based Clusters (vLCM-enabled clusters).

✅ Ensure that NSX and other solution components are managed by their respective life cycle tools.

Do not.

❌ Do not enable LCM Advanced Mode prior to or during vLCM enablement.

• If lcmProperties.upgrade.advanced.mode.on=true, third-party VIBs will be removed during the next upgrade cycle without warning.

❌ Do not install VIBs manually using esxcli software vib install.

• Manually installed VIBs are not part of the cluster image and will be removed in the next upgrade.

❌ Do not upload third-party solution depots by the ADD COMPONENTS feature in the VxRail plugin.

• This will negatively impact the cluster’s wanted state and block future upgrades.

❌ Do not use VMware vCenter Lifecycle Manager UI to make changes.

• Any modifications made here are not reflected in VxRail Manager, leading to life cycle inconsistencies.

❌ Do not reset the VMware vCenter Update Manager (VUM) database unless necessary.

• Resetting the database in a linked vCenter environment removes all VxRail depots and break compliance checks.
• This causes upgrade failures for all clusters attached to the vCenter.

❌ Do not edit or remove any folder or file from /data/store2/vlcm, /data/store2/hsm. 

• Uploading the composite bundle re-creates /data/store2/hsm and /data/store2/vlcm along with everything in them.

❌ Do not attempt to upgrade any third-party components outside of a VxRail vLCM upgrade cycle.

• This is entirely unsupported and breaks image compliance
• This does not apply to third-party solutions, as described above

❌ Do not use python CVSAllowListMgr.py (kb 185052) on VxRail vLCM Enabled Cluster.

• CVSAllowListMgr.py cannot be used for vLCM.
• Any attempt to use this script will break future upgrades.

❌Do not reference any Broadcom Knowledge Base articles.

• Note that most of Broadcom knowledge base articles referring to Lifecycle Manager do not apply to VxRail, and following them will result in additional complications on the VxRail side. 

Impact of Resetting the VUM Database

• Resetting the VUM database affects all linked vCenters and clusters.
• After a reset, all VxRail depots are removed and will not be automatically readded.
• Clusters enter a "host status unknown" state, blocking compliance checks and upgrades.
• Third-party solution depots (NSX, Rubrik) are removed, requiring manual readdition.
• If a VUM reset is necessary, it must be reviewed and approved by VxRail EE

Conclusion

Proper use of vLCM ensures a smooth upgrade and management process for VxRail environments. Following these guidelines help prevent upgrade failures, maintain compliance, and keep the system running optimally.