Dell VxRail: vSAN RDMA is Not Supported in VxRail 7.0.37x and 7.0.38x Releases

Cannot enable vSAN RDMA for VxRail clusters running on 7.0.37x and 7.0.38x releases.
vCenter UI --> vSAN cluster --> Configure --> vSAN --> Services --> Network.
UI gives the message "RDMA is not supported when large-scale cluster support is enabled."
 

VxRail Cluster will enable large-scale cluster support by default after initial configuration.
It can be disabled from vCenter UI under vSAN cluster -->. Configure --> vSAN --> Services --> Advance Options.
  


However, VMware removed this option from vCenter 7.0 Update 3d, there is no UI interface to edit, and other methods such as SDK, CLI also do not work. For example, Advanced Options page cannot edit large cluster support anymore in vCenter 7.0 Update 3d.
 

So with large-scale cluster support enabled, it is not able to enable vSAN RDMA.

If a customer intends to enable vSAN RDMA, please avoid deploying or upgrading to VxRail 7.0.37x and 7.0.38x releases as ESXi 7.0 Update 3d and later has "large cluster support" enabled by default, and no option to disable it.

Furthermore, if "large cluster support" is disabled prior to 7.0.37x upgrade, it ends up with "large cluster support" enabled by ESXi therefore unable to enable vSAN RDMA as a consequence.


This issue is already fixed in vCenter and ESXi 70U3f release.
You can upgrade to VxRail 7.0.400 or higher versions, and run the below power shell commands to disable 'largeScaleClusterSupport'. After disabling this feature, you can enable RDMA.
 

pwsh> Connect-VIServer -Server <vCenter-IP>

pwsh> Set-VsanClusterConfiguration -Configuration <ClusterName> -LargeClusterSupported 0 

This issue is also documented in VMware KB 88621 (External Link)


Refer to VMware vSAN Network Design Guide. 
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vsan-planning.doc/GUID-031F9637-EE29-4684-8644-7A93B9FD8D7B.html

• All hosts must be on the same subnet. vSAN with RDMA supports up to 32 hosts. 

For more information about vSAN RDMA
https://blogs.vmware.com/virtualblocks/2021/03/24/vsan-7-update-2-rdma/ (External Link)