RecoverPoint for VMs: ESXi hostname's connection with RPA x is down
Summary: When the splitter loses connection to a RecoverPoint Appliance (RPA), it may not recover the path. This leaves a warning in RecoverPoint for Virtual Machines alerts: {ESXi hostname's} connection with RPA x is down. ...
Symptoms
In RecoverPoint for Virtual Machines (VMs) Plugin, Alerts and in output of CLI "get_system_status -n":
Splitters:
ERROR: {ESXi Host name}'s connection with RPA x is down
Splitter logs (/scratch/log/kdriver.log.00000xxxxx on the ESXi host) show degraded state:
2022/04/11 12:38:24.098 - #2 - 2100091/2100043 - KS: krnl:[12:38:23.089] 0/0 #1 - IOTracker_s_periodicCleanup: Total of 49 IOs are about to be timeouted. The First timeouted IO [msgId = 8074040] was 12182572 MicroSec in queue while 10000000 allowed krnl:[12:38:23.089] 0/0 #0 - IoEsx_ToKbox_v_isSucceeded_i: IO 0x4325d0e8xxxx Failed. Host_Status = TIMEOUT, Device_Status = TASK_ABORTED, Pluging_Status = 0, Scsu SN = 952, bytesXferred = 16384, lbc = 32 krnl:[12:38:23.089] 0/0 #1 - PathRpa_notify_error: Cluster(xxxxxxxxxxxxxxxx) Role(1) Kbox(xx.xx.xx.xx:5042 Vmk(None), bounded(No)) errors_count=1, change to DEGRADED ... krnl:[12:38:33.090] 0/0 #1 - PathRpa_notify_error: Cluster(xxxxxxxxxxxxxxxx) Role(1) Kbox(xx.xx.xx.xx:5042 Vmk(None), bounded(No)) state=DEGRADED errors_count=100 ... krnl:[12:38:48.093] 0/0 #1 - PathRpa_notify_error: Cluster(xxxxxxxxxxxxxxxx) Role(1) Kbox(xx.xx.xx.xx:5042 Vmk(None), bounded(No)) state=DEGRADED errors_count=200
Cause
There are different causes of the Splitter > RPA path disconnection, but in some of the conditions, the splitter is not able to recover the degraded path.
Resolution
Workaround:
Restart the driver process on the ESXi host:
- Log in to the host listed in the ERROR with
SSHas root user - Run command
pkill -9 kdriver
Sometimes a restart of the vRPA is required:
- Log in to the vRPA listed in the ERROR with
SSHas admin user - Select [5] Shutdown Reboot operations > [1] Reboot RPA
Long-term workaround:
RecoverPoint for Virtual Machines 5.3 SP4 Splitter contains the code to automatically recover from most of most errors.
Dell Technologies Engineering has also provided a hotfix Splitter VIB file that contains a script that can automatically recover from most errors. The main goal is to reduce the manual intervention required when the splitter disconnects.
Contact the Dell Technologies Support Center or your service representative to access the VIB. Reference this article ID.
Resolution:
Dell Technologies engineering is investigating this issue. Contact the Dell Technologies Support Center or your service representative for assistance and reference this article ID.