VxBlock: Potential VMFS-L Locker partition corruption on SD cards in ESXi 7.0 Video

Summary: Device disconnection happened on USB hardware on ESXi. This is seen on xHCI controller that, when commands fail, and USB bus reset happens after one retry it leads to all USB device (including USB SD card) reconnection. When USB boot device is reconnected, ESXi host may not be able to release path resource and considers a new device plugged in and gives a new path. From the ESXi host, it shows boot device as lost. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Instructions

Workaround for VMFS-L Locker partition corruption issue.

Duration: 04:35
Closed captions: English only or Available in multiple languages. Use the CC icon to change the closed caption language.

  • ESXi host boots off USB SD card.
  • Logs or boot up errors indicates that boot drive is corrupt.
  • The issue may cause purple diagnostic screen errors (PSOD) on ESXi hosts.
  • If related to this issue, the diagnostic message should indicate or imply boot disk failure.
  • The partition that is corrupt is the VMFS-L locker partition.
  • vmkwarning.log may show entries similar to:
2021-02-02T19:26:34 cpu23:3614392)ALERT: Bootbank cannot be found at path '/bootbank'
  • vobd.log may show entries similar to:
2021-02-02T19:26:34.577Z: [vmfsCorrelator] 2938471510366us: [esx.problem.vmfs.resource.corruptondisk] 5fecb7ff-8662d6b5-7732-24b6fde3ff81 LOCKER-5fecb7ff-8662d6b5-7732-24b6fde3ff81
2021-02-02T19:31:38.981Z: [vmfsCorrelator] 2938700315575us: [vob.vmfs.resource.corruptondisk] Volume 5fecb7ff-8662d6b5-7732-24b6fde3ff81 ("LOCKER-5fecb7ff-8662d6b5-7732-24b6fde3ff81") might be damaged on the disk. Resource cluster metadata corruption has been detected.
  • vmkernel.log may show entries similar to:
2021-02-02T19:57:01.265Z cpu4:2098636)WARNING: [type 6] Invalid clusterNum 2251799813685250.
2021-02-02T19:57:01.265Z cpu4:2098636)WARNING: Res3: 7148: Volume 5fecb7ff-8662d6b5-7732-24b6fde3ff81 ("LOCKER-5fecb7ff-8662d6b5-7732-24b6fde3ff81") might be damaged on the disk. Resource cluster metadata corruption has been detected.
2021-02-02T19:57:01.265Z cpu4:2098636)WARNING: FS3: 633: VMFS volume LOCKER-5fecb7ff-8662d6b5-7732-24b6fde3ff81/5fecb7ff-8662d6b5-7732-24b6fde3ff81 on mpx.vmhba32:C0:T0:L0:7 has been detected corrupted
2021-02-02T19:38:02.798Z cpu7:2097218)ScsiDeviceIO: 4062: Cmd(0x45ba922e3e80) 0x1a, CmdSN 0x1fea2a0 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x7 D:0x0 P:0x0 Invalid sense data: 0x0 0x0 0x0.
2021-02-02T19:38:02.935Z cpu14:2097225)ScsiDeviceIO: 4062: Cmd(0x45ba92263280) 0x1a, CmdSN 0x1fea2a8 from world 0 to dev "mpx.vmhba32:C0:T0:L0" failed H:0x7 D:0x0 P:0x0 Invalid sense data: 0xc7 0xf 0x43.

Note: The preceding log excerpts are only examples. Date, time, and environment variables may vary depending on your environment.
 

Solution:
This issue is addressed in VMware vSphere ESXi 7.0 U2c.
For more information, see ESXi 7.0 U2c release notes.
To download software, go to the Customer Connect Patch Downloads page. 
This is a known issue in VMware vSphere  ESXi 6.7.

Note: As a best practice, do not set dump partition on USB storage device and do not set USB devices under a heavy workload.

For more information, see VMware articles: Configuring ESXi coredump to file instead of partition and High frequency of read operations on VMware Tools image may cause SD card corruption.

There is no resolution, but there is a workaround.

Workaround:
If SD card is lower tolerant devices, reduce heavy access to SD cards by following the below steps.

The ToolsRamdisk advanced option tells ESXi to copy vmtools, which exists on the SD-Card, to RAMdisk. It mainly happens on VSAN or VDI environment. This creates the advanced option, giving it an accepted value of 0 (disable) or 1 (enable).
  • To do this, run the following command:
esxcfg-advcfg -A ToolsRamdisk --add-desc "Use VMware Tools repository from /tools ramdisk" --add-default "0" --add-type 'int' --add-min "0" --add-max "1"
esxcli system settings advanced set -o /UserVars/ToolsRamdisk -i 1
reboot
 
This sets the ToolsRamdisk option to enable. On bootup, ESXi checks the value of ToolsRamdisk and copy the files to the /tools ramdisk.
  • Run command:
esxcli system visorfs ramdisk list

This command shows the ramdisks that are created. In particular, the /tools ramdisk appears in this output, and it has a Used Value of 200 MB, so the tools have been copied to there successfully.

For more information, see VMware article: High frequency of read operations on VMware Tools image may cause SD card corruption. 

Future ESXi 7.0.x versions have this advanced option set automatically. See VMware article: ToolsRamdisk option is not available with ESXi 7.0.x releases.

This issue may occur on several releases of ESXi, however the likelihood of experiencing the behavior is higher on ESXi 7.0 due to some changes in the product that require better performance and endurance from the boot device as noted below:

Starting in ESXi 7.0, the boot partition is formatted as VMFS-L instead of FAT (previous releases) to improve I/O performance.

The version 7.0 Update 2 VMware ESXi Installation and Setup Guide, page 19, says that "As even read-only workloads can cause problems on low-end flash devices, you should install ESXi only on high-endurance flash media.

Affected Products

VxBlock and vBlock Systems Series
Article Properties
Article Number: 000191541
Article Type: How To
Last Modified: 25 Nov 2025
Version:  4
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.