VPLEX: NDU Precheck Errors Observed on the VPLEX Clusters after Code Upgrade
Summary: The purpose of this article is to clear the Nondisruptive Upgrade (NDU) precheck errors appearing on the VPLEX clusters after the code upgrade.
This article applies to
This article does not apply to
This article is not tied to any specific product.
Not all product versions are identified in this article.
Symptoms
After the upgrade of GeoSynchrony code to 6.1, running
Example:
ndu pre-check command on VPlexcli from one of the VPLEX clusters returns errors. No errors are found when NDU precheck is run from the other cluster.
Example:
From cluster-1:
service@<cluster-1>:~> vplexcli Trying ::1... Connected to localhost. Escape character is '^]'. VPlexcli:/> ndu pre-check Warning: During the NDU process, multiple directors will be offline for a portion of the time. This is non-disruptive but is dependent on a host-based multipathing solution being installed, configured, and operating on all connected hosts. Warning: Please run the "health-check --full" command to verify that the VPLEX is healthy. Analyzing system configuration: .DONE ================================================================================ Performing NDU pre-checks ================================================================================ Verify NDU is not in progress.. ERROR Verify that AIX initiators are configured in VPLEX.. OK Verify director communication status.. OK Verify management network redundancy.. OK Verify management network latency.. ERROR Verify time drift between directors and management server ERROR Verify firmware software version can be retrieved ERROR Verify sufficient disk space on the management server ERROR Verify sufficient disk space on directors ERROR Verify director SSD health ERROR Verify bios quiet mode enabled on the directors ERROR Verify sps conditioning status.. ERROR Verify intra-cluster communications connectivity.. OK Verify inter-director management connectivity status.. OK Verify directors have been commissioned.. OK Verify no unreachable or dead storage-volumes.. OK Verify no unhealthy virtual-volumes.. OK Verify distributed device settings.. OK Verify no unhealthy storage views.. OK Verify valid system configuration.. OK Verify valid metadata volume.. ERROR Verify metadata volume redundancy.. OK Verify all local-com ports have correct topologies.. OK Verify cluster status.. ERROR Verify and prepare directors for ndu.. ERROR Verify the response time of front-end switches.. OK Verify the response time of back-end switches.. OK Verify for potential slow I/O in the last 30 seconds.. ERROR Verify meta-volume backup configuration.. ERROR Verify metadata volume slot availability.. OK Verify RecoverPoint cluster registrations.. OK Verify no virtual volume expansion.. OK Verify if array aware operations are in progress.. ERROR Verify all back-end connections comply with LUN count restriction.. OK Verify inter-cluster communications connectivity.. OK Verify the remote management server version.. ERROR Verify cache-mode consistency for distributed virtual-volumes.. OK Verify clusters are joined.. OK Verify cluster witness state.. ERROR Verify sufficient disk space on the cluster witness server.. ERROR Verify there is no recent I/O aborts.. ERROR
Cause
After upgrading to GeoSynchrony 6.1, there may be missing details in the configuration file on the management server because of which the corresponding cluster returns errors in NDU precheck output.
Resolution
Step 1: Log in to
Example:
From cluster-2:
Step 3: The above two steps confirm that the issue is seen on cluster-1 alone. Close both the PuTTY sessions.
Step 4: Launch a new PuTTY session to cluster-1 management server with service account and restart the management-server services from the latest
Step 7: Connect to the cluster-1 management server and log in to
Example:
From cluster-1:
VPlexcli from Cluster-1 management server and issue ndu pre-check command. The following errors are seen during the NDU precheck.
Example:
From cluster-1:
Step 2: Log in to service@<cluster-1>:~> vplexcli Trying ::1... Connected to localhost. Escape character is '^]'. VPlexcli:/> ndu pre-check Warning: During the NDU process, multiple directors will be offline for a portion of the time. This is non-disruptive but is dependent on a host-based multipathing solution being installed, configured, and operating on all connected hosts. Warning: Please run the "health-check --full" command to verify that the VPLEX is healthy. Analyzing system configuration: .DONE ================================================================================ Performing NDU pre-checks ================================================================================ Verify NDU is not in progress.. ERROR Verify that AIX initiators are configured in VPLEX.. OK Verify director communication status.. OK Verify management network redundancy.. OK Verify management network latency.. ERROR Verify time drift between directors and management server ERROR Verify firmware software version can be retrieved ERROR Verify sufficient disk space on the management server ERROR Verify sufficient disk space on directors ERROR Verify director SSD health ERROR Verify bios quiet mode enabled on the directors ERROR Verify sps conditioning status.. ERROR Verify intra-cluster communications connectivity.. OK Verify inter-director management connectivity status.. OK Verify directors have been commissioned.. OK Verify no unreachable or dead storage-volumes.. OK Verify no unhealthy virtual-volumes.. OK Verify distributed device settings.. OK Verify no unhealthy storage views.. OK Verify valid system configuration.. OK Verify valid metadata volume.. ERROR Verify metadata volume redundancy.. OK Verify all local-com ports have correct topologies.. OK Verify cluster status.. ERROR Verify and prepare directors for ndu.. ERROR Verify the response time of front-end switches.. OK Verify the response time of back-end switches.. OK Verify for potential slow I/O in the last 30 seconds.. ERROR Verify meta-volume backup configuration.. ERROR Verify metadata volume slot availability.. OK Verify RecoverPoint cluster registrations.. OK Verify no virtual volume expansion.. OK Verify if array aware operations are in progress.. ERROR Verify all back-end connections comply with LUN count restriction.. OK Verify inter-cluster communications connectivity.. OK Verify the remote management server version.. ERROR Verify cache-mode consistency for distributed virtual-volumes.. OK Verify clusters are joined.. OK Verify cluster witness state.. ERROR Verify sufficient disk space on the cluster witness server.. ERROR Verify there is no recent I/O aborts.. ERROR
VPlexcli from cluster-2 management server and issue ndu pre-check. Observe that there are no errors seen when NDU precheck is issued from cluster-2.
Example:
From cluster-2:
service@<cluster-2>:~> vplexcli Trying ::1... Connected to localhost. Escape character is '^]'. VPlexcli:/> ndu pre-check Warning: During the NDU process, multiple directors will be offline for a portion of the time. This is non-disruptive but is dependent on a host-based multipathing solution being installed, configured, and operating on all connected hosts. Warning: Please run the "health-check --full" command to verify that the VPLEX is healthy Analyzing system configuration: .DONE ================================================================================ Performing NDU pre-checks ================================================================================ Verify NDU is not in progress.. OK Verify that AIX initiators are configured in VPLEX.. OK Verify director communication status.. OK Verify management network redundancy.. OK Verify management network latency.. OK Verify time drift between directors and management server.. OK Verify firmware software version can be retrieved.. OK Verify sufficient disk space on the management server.. OK Verify sufficient disk space on directors.. OK Verify director SSD health.. OK Verify bios quiet mode enabled on the directors.. OK Verify sps conditioning status.. OK Verify intra-cluster communications connectivity.. OK Verify inter-director management connectivity status.. OK Verify directors have been commissioned.. OK Verify no unreachable or dead storage-volumes.. OK Verify no unhealthy virtual-volumes.. OK Verify distributed device settings.. OK Verify no unhealthy storage views.. OK Verify valid system configuration.. OK Verify valid metadata volume.. OK Verify metadata volume redundancy.. OK Verify all local-com ports have correct topologies.. OK Verify cluster status.. OK Verify and prepare directors for ndu.. OK Verify the response time of front-end switches.. OK Verify the response time of back-end switches.. OK Verify for potential slow I/O in the last 30 seconds.. OK Verify meta-volume backup configuration.. OK Verify metadata volume slot availability.. OK Verify RecoverPoint cluster registrations.. OK Verify no virtual volume expansion.. OK Verify if array aware operations are in progress.. OK Verify all back-end connections comply with LUN count restriction.. OK Verify inter-cluster communications connectivity.. OK Verify the remote management server version.. OK Verify cache-mode consistency for distributed virtual-volumes.. OK Verify clusters are joined.. OK Verify cluster witness state.. OK Verify sufficient disk space on the cluster witness server.. OK Verify there is no recent I/O aborts.. OK
Step 4: Launch a new PuTTY session to cluster-1 management server with service account and restart the management-server services from the latest
mgmtServerBackup.tar file.
service@<cluster-1> sudo /opt/emc/VPlex/bin/VPlex-MS-backup --restart-services --restore /tmp/mgmtServerBackup-<SERIAL-NUMBER>.tar
Ignore any errors that appear.
If the
Step 5: Reboot the management-server using the command below.
mgmtServerBackup-<SERIAL_NUMBER>.tar file is not present in the /tmp folder of the management server of VPLEX, it can be fetched from the director by following article 119212. Ensure that the mgmtServerBackup.tar file is placed in the /tmp folder of the management server.
service@<cluster-1> sudo /sbin/rebootStep 6: The management server reboots, but allow 5-10 minutes before proceeding.
Step 7: Connect to the cluster-1 management server and log in to
VPlexcli. Issue ndu pre-check again to confirm that the errors are no longer present.
Example:
From cluster-1:
service@<cluster-1>:~> vplexcli Trying ::1... Connected to localhost. Escape character is '^]'. VPlexcli:/> ndu pre-check Warning: During the NDU process, multiple directors will be offline for a portion of the time. This is non-disruptive but is dependent on a host-based multipathing solution being installed, configured, and operating on all connected hosts. Warning: Please run the "health-check --full" command to verify that the VPLEX is healthy. Analyzing system configuration: .DONE ================================================================================ Performing NDU pre-checks ================================================================================ Verify NDU is not in progress.. OK Verify that AIX initiators are configured in VPLEX.. OK Verify director communication status.. OK Verify management network redundancy.. OK Verify management network latency.. OK Verify time drift between directors and management server.. OK Verify firmware software version can be retrieved.. OK Verify sufficient disk space on the management server.. OK Verify sufficient disk space on directors.. OK Verify director SSD health.. OK Verify bios quiet mode enabled on the directors.. OK Verify sps conditioning status.. OK Verify intra-cluster communications connectivity.. OK Verify inter-director management connectivity status.. OK Verify directors have been commissioned.. OK Verify no unreachable or dead storage-volumes.. OK Verify no unhealthy virtual-volumes.. OK Verify distributed device settings.. OK Verify no unhealthy storage views.. OK Verify valid system configuration.. OK Verify valid metadata volume.. OK Verify metadata volume redundancy.. OK Verify all local-com ports have correct topologies.. OK Verify cluster status.. OK Verify and prepare directors for ndu.. OK Verify the response time of front-end switches.. OK Verify the response time of back-end switches.. OK Verify for potential slow I/O in the last 30 seconds.. OK Verify meta-volume backup configuration.. OK Verify metadata volume slot availability.. OK Verify RecoverPoint cluster registrations.. OK Verify no virtual volume expansion.. OK Verify if array aware operations are in progress.. OK Verify all back-end connections comply with LUN count restriction.. OK Verify inter-cluster communications connectivity.. OK Verify the remote management server version.. OK Verify cache-mode consistency for distributed virtual-volumes.. OK Verify clusters are joined.. OK Verify cluster witness state.. OK Verify sufficient disk space on the cluster witness server.. OK Verify there is no recent I/O aborts.. OK
Affected Products
VPLEX Series, VPLEX VS2, VPLEX VS6Article Properties
Article Number: 000168180
Article Type: Solution
Last Modified: 18 ذو القعدة 1447
Version: 4
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.