VNX: Replication Stops After Failover with "Warning 17455382845: Network communication has been lost"

Replication stops after Data Mover server_2 fails over to server_3 with the following error:

Warning 17455382845: Network communication has been lost

• Interconnect validation fails:

  nas_cel -interconnect -validate Replication_DR
  Error 12202: Interconnect validate failed between source interface and destination inestination interface=10.1.XX.46, Message_ID=13160415819: Unable to connect to destination

• The source interface can ping the destination interface.
• The DR is unable to ping the source, but the default gateway is pingable:

  server_ping server_2 10.1.XX.46
  server_2 :
  Error 6: server_2 : No such device or address
  no answer from 10.1.XX.46
  
  server_route server_2 -list
  server_2 :
  default 10.1.XX.1 0.0.0.0 Replication_DR
  net 128.XXX.253.0 128.XXX.253.2 255.XXX.255.0 el31
  net 128.XXX.252.0 128.XXX.252.2 255.XXX.255.0 el30
  net 10.1.XX.0 10.1.XX.15 255.XXX.255.0 Replication_DR
  net 10.0.0.0 10.1.XX.23 255.0.0.0 FILE-SRV04_INT
  host 127.X.X.1 127.X.X.1 255.XXX.255.255 loop
  
  server_ping server_2 10.1.XX.1
  server_2 : 10.1.XX.1 is alive, time= 0 ms
  
  nas_cel -interconnect -info -all --> from Secondary site
  
  id                                 = 20003
  name                               = Replication_DR
  source_server                      = server_2
  source_interfaces                  = 10.1.XX.15
  destination_system                 = JO00-VNX5400-CS0
  destination_server                 = server_2
  destination_interfaces             = 10.1.XX.46
  bandwidth schedule                 = uses available bandwidth
  crc enabled                        = yes
  number of configured replications  = 28
  number of replications in transfer = 0
  status                             = The interconnect is OK.

   

The default gateway of the secondary array is unable to reach the source interface after the Data Mover has failed over or rebooted.

Add a static route for the DR interface to reach the primary:

server_route server_2 -add host <dest> <gateway>
server_route server_2 -add host 10.1.XX.46 XX.1.XX.1
server_2 : done

Ping the primary to test the connection:

server_ping server_2 10.1.XX.46
server_2 : 10.1.XX.46 is alive, time= 0 ms

Validate the interconnect:

nas_cel -interconnect -validate Replication_DR
Replication_DR: has 1 source and 1 destination interface(s); validating - please wait...ok