VxRail: Pre checks fail when shutting down the entire VxRail cluster PrecheckException

Summary: Error message pre-checks fail when shutting down the entire VxRail appliance cluster pre-check PrecheckException.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

VxRail appliance shutdown
VxRail final pre-check failed due to internal errors.

During a full VxRail appliance shutdown, all the vCenter actions were timing out. Disabling HA, migrating the service VMs, so forth

Marvin.log 
2018-09-13T22:21:52:702Z [ERROR] [MessageBroker-2] o.s.s.s.TaskUtils$LoggingErrorHandler:95 - Unexpected error occurred in scheduled task.
2018-09-13T22:34:22:028Z [ERROR] [localhost-startStop-1] c.v.m.c.s.m.PlatformInformationService:252 - Unable to determine platform type: Failed to find primary host
2018-09-13T22:34:23:254Z [ERROR] [localhost-startStop-1] c.v.m.c.s.m.PlatformInformationService:252 - Unable to determine platform type: Failed to find primary host
2018-09-13T22:34:23:492Z [ERROR] [localhost-startStop-1] c.v.m.c.s.m.PlatformInformationService:252 - Unable to determine platform type: Failed to find primary host
Connectors-cluster.log 
Sep 13 22:31:19,167 GMT 2018  ERROR [AMQP Connection 127.0.0.1:5672] CachingConnectionFactory$DefaultChannelCloseLogger.log:1089 - Channel shutdown: connection error
Sep 13 22:31:19,167 GMT 2018  ERROR [AMQP Connection 127.0.0.1:5672] CachingConnectionFactory$DefaultChannelCloseLogger.log:1089 - Channel shutdown: connection error
Sep 13 22:31:19,167 GMT 2018  ERROR [AMQP Connection 127.0.0.1:5672] CachingConnectionFactory$DefaultChannelCloseLogger.log:1089 - Channel shutdown: connection error
Sep 13 23:13:00,858 GMT 2018  ERROR [SimpleAsyncTaskExecutor-1] PrecheckMQListener.check:66 - com.emc.mystic.manager.commons.emc.precheck.exception.PrecheckException

Sep 14 00:17:12,499 GMT 2018  ERROR [pool-51-thread-1] ServiceUtil.waitForTask:466 - Message The operation is not allowed in the current state.
Sep 14 00:17:12,499 GMT 2018  ERROR [pool-51-thread-1] VimOperator.migrateVMs:2016 - Migrating VM fails: vm-14
Sep 14 00:17:12,499 GMT 2018  ERROR [pool-51-thread-1] VimOperator.migrateVMs:2020 - Migrate VM failed:com.vmware.vim25.ManagedObjectReference@ 36891641  
java.lang.RuntimeException: Migrating VM fails: vm-14
 at com.emc.mystic.manager.commons.vmware.VimOperator.migrateVMs(VimOperator.java:2017) [mystic.manager.commons.vmware-4.0.401.jar:?]
 at com.emc.mystic.manager.connectors.cluster.ShutdownStartupServiceImpl$1.run(ShutdownStartupServiceImpl.java:162) [mystic.manager.connectors.cluster-4.0.401-lib.jar:?]
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_141]
 at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_141]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_141]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
Sep 14 00:17:12,500 GMT 2018  ERROR [pool-51-thread-1] ShutdownStartupServiceImpl$1.run:197 - Shutdown error
com.emc.mystic.manager.connectors.cluster.ClusterShutdownException: java.lang.RuntimeException: Migrating VM fails: vm-14
 at com.emc.mystic.manager.connectors.cluster.ClusterShutdownException.getClusterShutDownException(ClusterShutdownException.java:56) ~[mystic.manager.connectors.cluster-4.0.401-lib.jar:?]
 at com.emc.mystic.manager.connectors.cluster.ShutdownStartupServiceImpl$1.run(ShutdownStartupServiceImpl.java:164) [mystic.manager.connectors.cluster-4.0.401-lib.jar:?]
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_141]
 at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_141]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_141]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
Caused by: java.lang.RuntimeException: Migrating VM fails: vm-14
 at com.emc.mystic.manager.commons.vmware.VimOperator.migrateVMs(VimOperator.java:2017) ~[mystic.manager.commons.vmware-4.0.401.jar:?]
 at com.emc.mystic.manager.connectors.cluster.ShutdownStartupServiceImpl$1.run(ShutdownStartupServiceImpl.java:162) ~[mystic.manager.connectors.cluster-4.0.401-lib.jar:?]
 ... 5 more

Cause

vCenter is unable to function due to a lack of DNS.  In this customer's environment the DNS VMs are on the VxRail appliance as a normal VM.

Resolution

Workaround: Power one of the DNS server VMs on, and retry Shut it down manually after all other VMs are shut down.  
Alternatively, point the vCenter appliance to a DNS server not on the VxRail.

Additional Information

Recommendations: VxRail vCenter appliance should have it is DNS pointing to a server not hosted on the VxRail cluster.
If not, the DNS VM should be one of the last VMs to be shut down and should be placed on the last ESXi server to be shut down.

A customer also came across this issue with the Log Insight VM still running. 
Sep 13 23:46:57,625 GMT 2018  ERROR [pool-49-thread-1] ServiceUtil.waitForTask:466 - Message A general system error occurred: PBM error occurred during PreMigrateCheckCallback: Operation timed out
Sep 13 23:46:57,625 GMT 2018  ERROR [pool-49-thread-1] VimOperator.migrateVMs:2016 - Migrating VM fails: vm-17
Sep 13 23:46:57,625 GMT 2018  ERROR [pool-49-thread-1] VimOperator.migrateVMs:2020 - Migrate VM failed : com.vmware.vim25.ManagedObjectReference6f7a5a92
java.lang.RuntimeException: Migrating VM fails: vm-17
 at com.emc.mystic.manager.commons.vmware.VimOperator.migrateVMs(VimOperator.java:2017) [mystic.manager.commons.vmware-4.0.401.jar:?]
 at com.emc.mystic.manager.connectors.cluster.ShutdownStartupServiceImpl$1.run(ShutdownStartupServiceImpl.java:162) [mystic.manager.connectors.cluster-4.0.401-lib.jar:?]
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_141]
 at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_141]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_141]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
Sep 13 23:46:57,626 GMT 2018  ERROR [pool-49-thread-1] ShutdownStartupServiceImpl$1.run:197 - Shutdown error
com.emc.mystic.manager.connectors.cluster.ClusterShutdownException: java.lang.RuntimeException: Migrating VM fails: vm-17
 at com.emc.mystic.manager.connectors.cluster.ClusterShutdownException.getClusterShutDownException(ClusterShutdownException.java:56) ~[mystic.manager.connectors.cluster-4.0.401-lib.jar:?]
 at com.emc.mystic.manager.connectors.cluster.ShutdownStartupServiceImpl$1.run(ShutdownStartupServiceImpl.java:164) [mystic.manager.connectors.cluster-4.0.401-lib.jar:?]
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_141]
 at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_141]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_141]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_141]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
Caused by: java.lang.RuntimeException: Migrating VM fails: vm-17
 at com.emc.mystic.manager.commons.vmware.VimOperator.migrateVMs(VimOperator.java:2017) ~[mystic.manager.commons.vmware-4.0.401.jar:?]
 at com.emc.mystic.manager.connectors.cluster.ShutdownStartupServiceImpl$1.run(ShutdownStartupServiceImpl.java:162) ~[mystic.manager.connectors.cluster-4.0.401-lib.jar:?]
 ... 5 more

Affected Products

VxRail 460 and 470 Nodes

Products

VxRail 460 and 470 Nodes
Article Properties
Article Number: 000054970
Article Type: Solution
Last Modified: 05 Sep 2025
Version:  7
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.