VPLEX: Distributed Virtual Volumes Service Status Shows as Disconnected

Impacted VPLEX GeoSynchony code versions:
Software: GeoSynchony 6.0 Service Pack 1 Patch 2

Impacted VPLEX Hardware:
VPLEX Series
VPLEX VS2
VPLEX VS6
VPLEX-Metro

After an upgrade to GeoSynchrony 6.0 Service Pack 1 Patch 2 (6.0.1.02.00.03), distributed virtual volumes have a service status of "disconnected." This does not affect I/O, and the customer should not see DU due to this issue.

The AMF_REEXPORT_QD flag is a newly introduced flag in the firmware. It helps internally keep track of outstanding exports in the case that the AMG master goes down. It is implemented in amf_flags, which is a field that is visible to the CLI.

When the second upgraders come up during NDU, this flag is set as a part of normal operation. Typically, the flag would be unset during the re-export of the AMF. However, when booting, the first re-export skips this phase. This causes the flag to remain set until the next re-export. The flag itself is benign and does not interfere with I/O or system configuration.

When virtual-volume summary is run, the volumes have a service status of "disconnected":

Virtual-volume health summary

virtual-volume name                health state  operational status  service status
---------------------------------  ------------  ------------------  --------------
<vvol name>                         ok            ok                  disconnected

The underlying distributed device can also show as disconnected or running.

Example of a distributed device showing as "disconnected":

/distributed-storage/distributed-devices/<distributed-device name>:

Attributes:
Name                    Value
----------------------  --------------------------------
application-consistent  false
auto-resume             true
block-count             536870912
block-size              4K
capacity                2T
clusters-involved       [cluster-1, cluster-2]
geometry                raid-1
health-indications      []
health-state            ok
locality                distributed<<--
operational-status      ok
rebuild-allowed         true
rebuild-eta             -
rebuild-progress        -
rebuild-status          done
rebuild-type            full
rule-set-name           cluster-1-detaches

service-status      -->>disconnected<<--

stripe-depth            -
system-id               <name>
thin-capable            true
transfer-size           128K
virtual-volume          <vvol name>

Example of a distributed device showing as "running":

/distributed-storage/distributed-devices/<distributed-device name>:

Attributes:
Name                    Value
----------------------  ------------------------------------------
application-consistent  false
auto-resume             true
block-count             536874240
block-size              4K
capacity                2T
clusters-involved
consistency-group
geometry                raid-1
health-indications      []
health-state            ok
locality                distributed
operational-status      ok
rebuild-allowed         true
rebuild-eta             -
rebuild-progress        -
rebuild-status          done
rebuild-type            full
rule-set-name           cluster-2-detaches
service-status          running        <<<<
storage-array-family    symmetrix
stripe-depth            -
system-id               <name>
thin-capable            true
transfer-size           128K
virtual-volume          <vvol name>

A bug in the server-side FLL code is causing the reexport-qd flag to be read from the AMF. The code actually intends to read the non-splitting-fail-all flag. This causes the CLI to report the volume as disconnected when querying with FLL.

FLL is a feature introduced in GeoSynchrony 6.0 Service Pack 1 Patch 2 (6.0.1.02.00.03). In this feature, some management server tasks are offloaded onto the directors to alleviate possible performance impact to the management server.

If this issue is observed after an upgrade to GeoSynchrony 6.0 Service Pack 1 Patch 2 (6.0.1.02.00.03), contact Dell Customer Support and quote this Knowledge Base article ID.