VPLEX: Cannot enable a VS2 Front-End (FE) Port

Impact:
The inability to enable a VS2 front-end port in VPlexcli can lead to prolonged loss of redundancy on the Front End (FE) of the VPLEX, and also has the potential to impact a Generational Hardware Upgrade (GenU) of a VS2 to VS6.

Issue:
A VS2 running GeoSynchrony 6.x has a front-end (FE) port listed in the VPlexcli as having a status of "down" indicating it is disabled.

Attempts to reenable the FE port in VPlexcli (using the "set enabled true" command) do not return any errors, yet the FE port remains in "down" state.

Example for verifying the issue:

1. Check the FE port status by using the following command, and check what FE ports are down (disabled):

   In the following sample output below (we are only including the output from director-1-1-B):

   • Director-1-1-B B0-FC02 is "down" (disabled)
   • This is the port with the issue

   Also note:

   • Ports B0-FC00 and FC01 are "up" (enabled and have connectivity)
   • Port B0-FC03 is showing as "no-link" state (enabled, up, yet no connectivity, meaning no light level from the source end)

   Checking the status of the ports on all engines and ports

   VPlexcli:/> ll /engines/*/directors/*/hardware/ports*
   or
   VPlexcli:/> ll /engines/**/ports
   
   /engines/engine-1-1/directors/director-1-1-B/hardware/ports:
   
   Name     Address             Role       Port Status
   -------  ------------------  ---------  -----------
   B-CMI00  2                   -          down
   B0-FC00  0x50001442902a0a00  front-end  up
   B0-FC01  0x50001442902a0a01  front-end  up
   B0-FC02  0x0000000000000000  front-end  down <<<< port with issue
   B0-FC03  0x50001442902a0a03  front-end  no-link
   B1-FC00  0x50001442902a0a10  back-end   up

2. Attempt to enable the "down" port by using the VPlexcli command: set enabled true

   1. First, cd into the context of the FE port in question,
      Example:

      VPlexcli:/> cd /engines/engine-1-1/directors/director-1-1-B/hardware/ports/B0-FC02

   2. Then issue the command: set enabled true
      Example:

      VPlexcli:/engines/engine-1-1/directors/director-1-1-B/hardware/ports/B0-FC02> set enabled true
      VPlexcli:/engines/engine-1-1/directors/director-1-1-B/hardware/ports/B0-FC02>

      Note: No error will be returned, after the command is run, the prompt returns as shown above.
   3. From the "B0-FC02" context level, issue the "cd .." command to change to the "ports" context level of the director, and then run the long listing, "ll" command to check the state of the port again. It still shows with the "down" status:
      Example:

      VPlexcli:/engines/engine-1-1/directors/director-1-1-B/hardware/ports/B0-FC02> cd ..
      VPlexcli:/engines/engine-1-1/directors/director-1-1-B/hardware/ports> ll
      Name     Address             Role       Port Status
      -------  ------------------  ---------  -----------
      B-CMI00  2                   -          down
      B0-FC00  0x50001442902a0a00  front-end  up
      B0-FC01  0x50001442902a0a01  front-end  up
      B0-FC02  0x0000000000000000  front-end  down <<< still shows as down like before
      B0-FC03  0x50001442902a0a03  front-end  no-link

A VS2 to VS6 GenU can also fail if a VS2 has a FE port in the "down" state prior to the GenU.

During the GenU, the state/settings of the VS2 (including that of the FE ports) are transferred to the VS6, and then during the I/O Transfer Phase portion of the GenU, when the VS6 director is being brought up, the FE ports are enabled. If one or more VS2 FE ports are in a "down" state (disabled) prior to the GenU, and especially if those ports cannot be enabled in the VPlexcli, it can lead to the I/O Transfer Phase failing. As the step of enabling the FE ports on the VS6 director silently fails, and then a subsequent step waiting for all FE ports to have the "enabled"; state (on the VS6 director) fails due to a timeout error.

Example of the error returned during the I/O Transfer Phase of the GenU:

Continuing I/O transfer with connectivity issues could cause data unavailability and system instability. Connectivity issues must be fixed before proceeding. Would you like to re-check connectivity? (Retry/Abort): retry
Transferring director-2-1-A to VS6: .....................ERROR <<<<<
Failed to complete director transfer to VS6: hardware-upgrade Evaluation of
 <<hardware-upgrade transfer-io --vs6 --director transfer-io: director-2-1-A>> failed. <<<<<<
cause: Command execution failed. cause: *Evaluation of <<wait -c
[/engines/engine-2-1/directors/director-2-1-A/hardware/ports/A0-FC00,
 /engines/engine-2-1/directors/director-2-1-A/hardware/ports/A0-FC01,
 /engines/engine-2-1/directors/director-2-1-A/hardware/ports/A0-FC02,
 /engines/engine-2-1/directors/director-2-1-A/hardware/ports/A0-FC03] -a enabled -v*
 True -t 240>> failed. cause: Command execution failed. cause: Timed out after 240 seconds

An internal VPLEX component related to the FE port is in a disabled state. The root cause of why this state becomes disabled is not yet known.

If this issue is encountered engage Dell VPLEX Customer Service to implement a workaround to enable the FE port. When contacting Dell Customer Service, mention this article.

Permanent Fix:
Dell VPLEX Engineering is currently investigating this problem. Once a fix is available, this article will be updated.