Dell Unity: Storage Server Detected an Internal Failure in the VMware Hypervisor. Error Code:0x5c0000c
Summary: The storage server detected an internal failure in the VMware hypervisor. Please review the event logs on the VMware hypervisor side. (Error Code:0x5c0000c)
Symptoms
Customer Environment:
Dell Technologies Unity Array
ESXi Hypervisor - Free edition
(No vSphere servers)
When provisioning VMware data store from Array (Block DataStore), the creation failed with the following error:
The storage server detected an internal failure in the VMware hypervisor. Please review the event logs on the VMware hypervisor side. (Error Code:0x5c0000c)
Creation of the Lun is successful, and the customer can still scan and add manually the Lun to a datastore.
The same error is also reported when deleting the datastore from Unity.
Cause
This is not an issue with Array. ESXi free edition does not allow any modifications over VAAI. It allows "read-only" operations only.
The above error might be reported for NAS NFS operations too and might also be reported during datastore delete operations too, due to the same reason.
Resolution
Verify the ESXi license. If it does not have a paid license, it blocks any operations like "Creating Datastore." ESXi free edition permits "read-only" operations only.
To verify this, on the ESXi host, verify for "vim.fault.RestrictedVersion" message on the /var/log/hostd.log.
Example from a customer box (10.x.x.1xx is Unity IP in this example and 10.x.x.2xx is the esxi hypervisor IP):
2016-09-01T00:02:04.043Z [31380B70 info 'Vimsvc.ha-eventmgr' opID=hostd-fcae] Event 365 : User root@10.0.0.xxx logged in as gSOAP/2.8 2016-09-01T00:02:04.058Z [31380B70 verbose 'Default' opID=EB2968DD-00001161 user=root] AdapterServer: target='vmodl.query.PropertyCollector:ha-property-collector', method='waitForUpdates' 2016-09-01T00:02:04.065Z [31D80B70 info 'Vimsvc.TaskManager' opID=hostd-81d8 user=root] Task Created : haTask--vim.SearchIndex.findByIp-40794283 2016-09-01T00:02:04.066Z [31D80B70 verbose 'Solo.SearchIndex' opID=hostd-81d8 user=root] FindByAttribute: dcRef = <null>, attrVal = 10.x.x.2xx, vmSearch = false 2016-09-01T00:02:04.071Z [31D80B70 verbose 'Solo.SearchIndex' opID=hostd-81d8 user=root] FindByAttribute: returning vim.HostSystem:ha-host 2016-09-01T00:02:04.071Z [31D80B70 info 'Vimsvc.TaskManager' opID=hostd-81d8 user=root] Task Completed : haTask--vim.SearchIndex.findByIp-40794283 Status success 2016-09-01T00:02:04.156Z [2F640B70 info 'Solo.Vmomi' opID=hostd-e2e2 user=root] Activation [N5Vmomi10ActivationE:0x30997200] : Invoke done [rescanHba] on [vim.host.StorageSystem:storageSystem] 2016-09-01T00:02:04.156Z [2F640B70 verbose 'Solo.Vmomi' opID=hostd-e2e2 user=root] Arg hbaDevice: --> "vmhba33" 2016-09-01T00:02:04.156Z [2F640B70 info 'Solo.Vmomi' opID=hostd-e2e2 user=root] Throw vim.fault.RestrictedVersion 2016-09-01T00:02:04.156Z [2F640B70 info 'Solo.Vmomi' opID=hostd-e2e2 user=root] Result: --> (vim.fault.RestrictedVersion) { --> dynamicType = <unset>, --> faultCause = (vmodl.MethodFault) null, --> msg = "", --> } 2016-09-01T00:02:04.174Z [31380B70 verbose 'Default' opID=hostd-2bc7 user=root] CloseSession called for session id=c3304bfa-1fb1-10d4-9873-259b210357ea 2016-09-01T00:02:04.175Z [31380B70 info 'Vimsvc.ha-eventmgr' opID=hostd-2bc7 user=root] Event 366 : User root@10.x.x.xxx logged out (login time: Thursday, 01 September, 2016 00:02:03, number of API invocations: 0, user agent: gSOAP/2.8)
So the above log indicates, Unity logged in and received the vim.fault.RestrictedVersion error, so it logged out without completing the "datastore" creation and reports "Error Code:0x5c0000c"
Customer must pay for license/vCenter servers Or Provision the data store manually.