Dell VPLEX: Is it safe to disable WAN-COM ports
Summary: This article addresses the question about "if it is safe to disable WAN-COM ports" and explains how to safely do this without disrupting production.
Instructions
Issue:
There may be are problems experienced within VPLEX or external to VPLEX that requires disabling one or more VPLEX WAN-COM ports.
Cause:
No direct cause. The decision is determined from VPLEX administrator's conclusion of analysis of the issue.
Instructions:
If there are problems being encountered through the VPLEX WAN-COM that requires ports to be disabled, it is important to understand the end goal from disabling these ports and which directors are involved.
First, it is important to check if there are any pre-existing issues on the VPLEX WAN-COM by running the VPlexcli command 'connectivity validate-wan-com'.
Example of healthy WAN-COM connectivity on VPLEX Metro with Fiber Channel WAN:
VPlexcli:/> connectivity validate-wan-com connectivity: FULL fc-port-group-2-0 - OK - All expected connectivity is present. fc-port-group-2-1 - OK - All expected connectivity is present.
Example of healthy WAN-COM connectivity on VPLEX Metro with IP WAN:
VPlexcli:/> connectivity validate-wan-com connectivity: FULL ip-port-group-0 - OK - All expected connectivity is present. ip-port-group-1 - OK - All expected connectivity is present.
Port-Group-0 is defined as a full connectivity of all WAN-00 ports such as A2-FC00/B2-FC00 on Metro FC, and A2-XG00/B2-XG00 on Metro IP. All ports must be up and communicating in order to form a quorum on the port-group.
Port-Group-1 is defined as a full connectivity of all WAN-01 ports such as A2-FC01/B2-FC01 on Metro FC, and A2-XG01/B2-XG01 on Metro IP. All ports must be up and communicating in order to form a quorum on the port-group.
Second, if you must disable WAN-COM ports on VPLEX and do not have a healthy connectivity status prior to doing so, engage VPLEX support for assistance.
Third, it is important to be aware that if ports on all configured port-groups on the VPLEX WAN-COM are disabled simultaneously this triggers an outage between the two clusters. When this disruption occurs it triggers a WAN-COM outage which invokes the detach-rules on distributed-devices and if there are consistency-groups (CGs) the winner rules are also triggered, since the two clusters no longer be able to form a quorum over the WAN-COM.
Fourth, if there is a need to disable both WAN-COM ports on a single director, for example A2-FC00 and A2-FC01, but the desire is to not cause the two VPLEX clusters from separating, engage VPLEX Support to shutdown the firmware on the director. This allows the wanted ports to be taken offline while the underlying problem can be investigated and repaired, without triggering the detach-rules on the distributed-devices or winner-rules on any CGs.
The director when taken offline is removed from the quorum which allows normal functionality to proceed on the remaining directors.
How to disable VPLEX WAN-COM ports to invoke a full separation between the two clusters:
This can only be done in the VPlexcli.
-
Open a putty session to the VPLEX management-server, then launch ' vplexcli ' to connect to the command console.
-
Run the following command to disable all configured WAN-COM ports on cluster-1:
VPlexcli:/> set /clusters/cluster-1/connectivity/wan-com/port-groups/*::enabled all-disabled
-
Allow 4-5 seconds before verifying that 'connectivity' shows 'none' with the following command:
connectivity validate-wan-comExample:
VPlexcli:/> connectivity validate-wan-com connectivity: NONE <-- fc-port-group-2-3 - FAIL - No connectivity was found from any com port. fc-port-group-2-0 - FAIL - No connectivity was found from any com port. fc-port-group-2-2 - FAIL - No connectivity was found from any com port. fc-port-group-2-1 - FAIL - No connectivity was found from any com port.
-
Once repair efforts are complete, use the following actions to reenable the WAN-COM ports:
VPlexcli:/> set /clusters/cluster-1/connectivity/wan-com/port-groups/*::enabled all-enabled
-
Allow 4-5 seconds before verifying that 'connectivity' shows 'FULL' with the following command:
connectivity validate-wan-comExample:
VPlexcli:/> connectivity validate-wan-com connectivity: FULL <-- fc-port-group-2-3 - OK - All expected connectivity is present. fc-port-group-2-0 - OK - All expected connectivity is present. fc-port-group-2-2 - OK - All expected connectivity is present. fc-port-group-2-1 - OK - All expected connectivity is present.
How to disable VPLEX WAN-COM ports without causing an outage between the two clusters:
Be aware of the status of the VPLEX WAN-COM before proceeding with disabling individual WAN ports. If your 'connectivity validate-wan-com' output indicates anything other than 'FULL', engage VPLEX Support and do not proceed below.
Ensure that you are only disabling ports on a single port-group, and do not disable ports on multiple port-groups simultaneously, in such a way that all WAN paths to any director are lost, as this causes the two clusters to depart.
-
Open a putty session to the VPLEX management-server, then launch 'vplexcli' to connect to the command console.
-
Determine which director and port you want to disable and CD into the context.
Example:
VPlexcli:/> cd /engines/<engine>/directors/<director>/hardware/ports/<Port>
For example, to disable A2-FC01 on director-2-2-A on VPLEX Cluster-2 use the following commands:
VPlexcli:/> cd /engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01 VPlexcli:/engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01> ll Name Value ------------------ ------------------ address 0x500144xxxxxxxxxx current-speed 8Gbits/s enabled true max-speed 8Gbits/s node-wwn 0x500144xxxxxxxxxx operational-status ok port-status up port-wwn 0x500144xxxxxxxxxx protocols [fc] role wan-com target-port - topology - VPlexcli:/engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01> set enabled false VPlexcli:/engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01> ll Name Value ------------------ ------------------ address 0x0000000000000000 current-speed - enabled false max-speed 0Gbits/s node-wwn 0x0000000000000000 operational-status stopped port-status down port-wwn 0x0000000000000000 protocols [fc] role wan-com target-port - topology -
-
After disabling any WAN-COM ports, ensure to validate the status of connectivity with the following command:
connectivity validate-wan-comExample:
VPlexcli:/engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01> connectivity validate-wan-com connectivity: PARTIAL fc-port-group-2-0 - OK - All expected connectivity is present. fc-port-group-2-1 - ERROR - Connectivity errors were found for the following com ports: /engines/engine-1-1/directors/director-1-1-A/hardware/ports/A2-FC01 -> Missing connectivity to /engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01 /engines/engine-1-1/directors/director-1-1-B/hardware/ports/B2-FC01 -> Missing connectivity to /engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01 /engines/engine-1-2/directors/director-1-2-A/hardware/ports/A2-FC01 -> Missing connectivity to /engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01 /engines/engine-1-2/directors/director-1-2-B/hardware/ports/B2-FC01 -> Missing connectivity to /engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01 /engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01 -> Missing all expected connectivity.
-
Notice that the connectivity status is now 'PARTIAL' instead of 'FULL' indicating there is a loss of quorum on a port-group reporting the ERROR.
-
To reenable the port, 'cd' (change directory), into the port context and run the following command as shown below:
set enabled trueVPlexcli:/> cd /engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01 VPlexcli:/engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01> ll Name Value ------------------ ------------------ address 0x0000000000000000 current-speed - enabled false max-speed 0Gbits/s node-wwn 0x0000000000000000 operational-status stopped port-status down port-wwn 0x0000000000000000 protocols [fc] role wan-com target-port - topology - VPlexcli:/engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01> set enabled true VPlexcli:/engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01> ll Name Value ------------------ ------------------ address 0x500144xxxxxxxxxx current-speed 8Gbits/s enabled true max-speed 8Gbits/s node-wwn 0x500144xxxxxxxxxx operational-status ok port-status up port-wwn 0x500144xxxxxxxxxx protocols [fc] role wan-com target-port - topology -
-
Allow 4-5 seconds before checking if there is improvement in 'connectivity validate-wan-com' as shown below:
VPlexcli:/engines/engine-2-2/directors/director-2-2-A/hardware/ports/A2-FC01> connectivity validate-wan-com connectivity: FULL fc-port-group-2-0 - OK - All expected connectivity is present. fc-port-group-2-1 - OK - All expected connectivity is present.