Dell Unity: Replication mode changes from sync to async after code upgrade to 5.2 (Dell Correctable)
Summary: The replication mode might get changed from sync to async after upgrading to code 5.2. This leads to replication connection validation failure and impact replication-related operations like pause/resume/failover/failback. ...
Symptoms
Sync replication sessions will not resume.
Replication connections on Source will be showing status "Updated Needed."
The connection type gets changed from sync to async as shown below from uemcli output:
uemcli -d <IP> -u service -p <password> /remote/sys show -detail
2: ID = RS_1 <<< RS stands for Remote System. Usually RS_0 is the local system, RS_1 is the first remote system that configures on the local Unity and RS_2 is the second Remote System.
Name = XXX-Unity
Address = 10.xxx.xxx.xxx
Alternate Management Address = 10.xxx.xxx.xxx
Model = Unity 480F
Serial number = CKM00XXXXXXXXX
Connection type = async <<<<<<<<<<<< the mode was sync before code upgrade
Source interfaces = 10.xxx.xxx.xxx, 10.xxx.xxx.xxx
Local interfaces = 10.xxx.xxx.xxx, 10.xxx.xxx.xxx
Remote interfaces = 10.xxx.xxx.xxx, 10.xxx.xxx.xxx
Operational status = Update Needed (0x8406)
Health state = Minor failure (15)
Health details = "Update the remote system connection to pick up the latest interface changes on the local and remote systems. (https://10.xxx.xxx.xxx/help/webhelp/en_US/index.html?#unity_t_configure_replication_connections.html)"
Synchronous FC ports = spb_iom_1_fc0, spa_iom_1_fc0
Bandwidth schedules = uses available bandwidth
Current bandwidth = uses available bandwidth
If customer tries to 'validate and update' the connection, the operation fails with error "No remote interfaces received from the remote system".
EMCsystemlog:
"2022-06-15T22:56:51.960Z" "XXX-Unity_spa@CKM00XXXXXXXXX" "Neo_CEM" "26592" "local/admin" "ERROR" "14:610012" :: "User local/admin failed to validate remote system connections RS_1." :: Category=Audit Component=ReplicationPlugin TimeZone=UTC
cemtracer_dataprotection.log:
15 Jun 2022 22:56:51 - [DataProtectionStorageModel] INFO - {0:271532:843299283}[26592|30974|d53ffb40][refreshLocalExternalSystemInterconnects @ ../../../components/providers/osls/common/DataProtection/StorageModel/impl/StorageResourceSubsystemImpl.cpp:1315] Updating remote system name to: XXX-Unity
15 Jun 2022 22:56:51 - [DataProtectionStorageModel] ERROR - {0:271533:341483364}[26592|30974|d53ffb40][refreshLocalExternalSystemInterconnects @ ../../../components/providers/osls/common/DataProtection/StorageModel/impl/StorageResourceSubsystemImpl.cpp:1334] No remote interfaces recieved from the remote system
15 Jun 2022 22:56:51 - [DataProtectionStorageModel] ERROR - {0:271533:343407806}[26592|30974|d53ffb40][refreshLocalExternalSystemInterconnects @ ../../../components/providers/osls/common/DataProtection/StorageModel/impl/StorageResourceSubsystemImpl.cpp:1425] Caught exception: StorageModel exception [105906509]: Failed to get the list of interfaces from the remote system.
Cause
Engineering has confirmed the cause for the connection type change from 'sync' to 'async' is a combination of the following two known behaviors in current code (5.2):
1. Since 5.1, replication connection operational status will show as "Update Needed (0x8406)" after SP reboot (NDU upgrade), this has been fixed in Unity OE version 5.2.1.0.5.013.
2. In current 5.2 release, modifying replication connection with Remote User Name and Password input (without connection type explicitly specified) from GUI or uemcli will end in replication connection type changed to 'async' (from 'sync' or 'both'), this is planned to be fixed in a future version.
So far the issue (replication mode changes from sync to async) is only observed on the source side of the Unity. If you see a DR Unity having same issue, please escalate the case to L2 and engage EE to further investigate the issue. The workaround in Resolution section of this Dell Knowledge Base article can still be used.
Resolution
The fix is available in OS 5.2.1.0.5.013 (5.2 SP1)
Workaround:
Workaround for replication connection type mismatch issue between two sides after upgraded to 5.2.
a. For replication connection with 'sync' type (Mismatch of site A is 'async' and site B is 'sync')
1> Change 'sync' type to 'both' from site B. (Now two sites are 'both' type)
2> Change 'both' type to 'sync' from site B (or site A).
b. For replication connection with 'both' type (Mismatch of site A is 'async' and site B is 'both')
1> Change 'both' type to 'async' from site B. (Now two sites are 'async' type)
2> Change 'async' type to 'both' from site B (or site A).
Note:
Below uemcli commands can be used to change the connection type to both or sync. The RS_x in -id field is the id of the remote system that having issue.
Change to both:
uemcli -u admin -p XXXXXX /remote/sys -id RS_1 set -connectionType bothChange to sync:
uemcli -u admin -p XXXXXX /remote/sys -id RS_1 set -connectionType sync
Command syntax:
[Set]
/remote/sys -id <value> set [ -addr <value> ] [ -dstUsername <value> { -dstPassword <value> | -dstPasswordSecure } ] [ -connectionType { sync | async | both } ] [ -bandwidthSchedules <value> ]
- Once the mode on both ends is changed to sync, please validate the connection on both ends one at a time.