VPLEX: Management server reporting directors unknown
Summary: After a power outage, Management server reboot, or if the Management Console Service is restarted while the VPN is down, VPLEX reports "unreachable directors"
Symptoms
After a power outage or a cluster reboot, the Management server may not be able to see the VPlex directors and is unable to communicate with them.
In a Metro configuration the Management server of one cluster would report all directors are up while the other one is reporting unknown.
VPlexcli:/> cluster status
WARNING: There are unreachable directors: director-2-1-A director-2-1-B..
Connectivity may still have errors even if none are reported.
WARNING: Did not find corresponding wan-com port-group 'ip-port-group-0' on
cluster 'cluster-2'.
WARNING: Did not find corresponding wan-com port-group 'ip-port-group-1' on
cluster 'cluster-2'.
Cluster cluster-1
operational-status: ok
transitioning-indications:
transitioning-progress:
health-state: ok
health-indications:
local-com: ok
wan-com: connectivity: NONE
ip-port-group-0 - FAIL - Failed to determine expected connectivity.
ip-port-group-1 - FAIL - Failed to determine expected connectivity.
Cause
After a Management server reboots or the Management Console Service is restarted while the VPN is down, the local vplexcli can sometimes lose connection with the remote and/or local directors and the directors need to be reconnected to the vplexcli.
Resolution
1. Confirm the VPN is up and the remote management server is reachable:
VPlexcli:/> vpn status
Verifying the VPN status between the management servers...
IPSEC is UP
Remote Management Server at IP Address xxx.xxx.xxx.xxx is reachable
Remote Internal Gateway addresses are reachable
Verifying the VPN status between the management server and the cluster witness server...
IPSEC is UP
Cluster Witness Server at IP Address 128.221.254.3 is reachable
2. Confirm that metadata-volumes are connected and no there are issues with the BE array:
VPlexcli:/> ll clusters/cluster-1/system-volumes/
/clusters/cluster-1/system-volumes:
Name Volume Type Operational Health Active Ready Geometry Component Block Block Capacity Slots
------------------------------------------ -------------- Status State ------ ----- -------- Count Count Size -------- -----
------------------------------------------ -------------- ----------- ------ ------ ----- -------- --------- -------- ----- -------- -----
LOGGING_VOL logging-volume ok ok - - raid-1 2 2621440 4K 10G -
LOGGING_VOL_2 logging-volume ok ok - - raid-1 2 5242880 4K 20G -
META_DATA meta-volume ok ok true true raid-1 2 20446976 4K 78G 64000
META_DATA_backup_2017Dec1 meta-volume ok ok false true raid-1 1 20446976 4K 78G 64000
META_DATA_backup_2017Dec1 meta-volume ok ok false true raid-1 1 20446976 4K 78G 64000
service@CLUSTER-2:~> ping 128.221.252.67
PING 128.221.252.67 (128.221.252.67) 56(84) bytes of data.
64 bytes from 128.221.252.67: icmp_seq=1 ttl=64 time=0.231 ms
64 bytes from 128.221.252.67: icmp_seq=2 ttl=64 time=0.189 ms
service@CLUSTER-2:~> ssh root@128.221.252.67
Last login: Mon Dec 11 16:58:29 2017 from 128.221.252.65
VPLEX
director-2-1-a:~ #
4. List out the engines on the remote cluster as you will be prompted for the number of engines when you run the "configuration connect-remote-directors" command
VPlexcli:/> ll /engines/
/engines:Name----------
engine-2-1
VPlexcli:/> configuration connect-local-directors
Connected to Plex firmware director-1-1-A.
Connected to Plex firmware director-1-1-B.
VPlexcli:/> configuration connect-remote-directors
Please enter the number of engines present on the remote site (1, 2 or 4): 1
Connected to Plex firmware director-2-1-A.
Connected to Plex firmware director-2-1-B.