PowerProtect: IDPA Hypervisor Manager Upgrade Failure - Missing Default Gateway on Old VCSA

Summary: Integrated Data Protection Appliance: 2.7.6 Hypervisor Manager Upgrade Failed at 25% Due to Default Gateway Field Missing on Old VCSA Before Upgrade.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

  1. The Hypervisor Manager workflow is failed at 25% component progress.

  2. The ACM primary upgrade log (/data01/tmp/patch/logs/upgrade-utility.log) shows that the "gateway" field (--prop:guestinfo.cis.appliance.net.gateway='') is blank during vCenter deployment causing the private IP to not be assigned after deployment.

    2024-01-31 23:39:33,854 INFO  [upgrade-workflow-3]-vi.UpgradeVijavaAccess: getAcmVM -->  ACM VM Name  DataProtection-ACM
2024-01-31 23:39:33,857 INFO  [upgrade-workflow-3]-vi.VCenterService: getCommandToDeployVCSA --> Command without conncetion properties timeout 30m ovftool --acceptAllEulas --noSSLVerify --sourceType=OVA --name=DataProtection-VCSA --datastore="vsanDatastore" --deploymentOption=tiny --diskMode=thick --allowExtraConfig --X:injectOvfEnv --X:enableHiddenProperties --X:waitForIp --net:"Network 1=DP-appliance-internal" --ipAllocationPolicy=fixedPolicy --ipProtocol=IPv4 --powerOn --prop:guestinfo.cis.deployment.autoconfig=True --prop:guestinfo.cis.appliance.root.passwd="***" --prop:guestinfo.cis.deployment.node.type=embedded --prop:guestinfo.cis.appliance.net.addr.family=ipv4 --prop:guestinfo.cis.appliance.net.addr=192.168.100.99 --prop:guestinfo.cis.appliance.net.prefix=27 --prop:guestinfo.cis.vmdir.domain-name=vSphere.local --prop:guestinfo.cis.appliance.net.gateway='' --prop:guestinfo.cis.appliance.ssh.enabled=True --prop:guestinfo.cis.appliance.net.mode=static --prop:guestinfo.cis.ceip_enabled=False --prop:guestinfo.cis.vmdir.site-name=Default-First-Site --prop:guestinfo.cis.appliance.time.tools-sync=True --prop:guestinfo.cis.vmdir.password="***" --prop:guestinfo.cis.appliance.net.dns.servers=192.168.100.100 --prop:guestinfo.cis.appliance.net.pnid=dpappliance-vcsa.idpa.local /data01/tmp/patch/products/VCSA/binaries/vcsa.ova
2024-01-31 23:39:33,857 INFO  [upgrade-workflow-3]-dao.InfrastructureComponentsDAOImpl: getInfrastructureInfo Retrieving infrastructure components details from file /usr/local/dataprotection/var/configmgr/server_data/config/InfrastructureComponents.xml
2024-01-31 23:39:33,858 INFO  [upgrade-workflow-3]-dao.InfrastructureComponentsDAOImpl: getInfrastructureInfo Successfully retrieved infrastructure components details
2024-01-31 23:39:33,866 INFO  [upgrade-workflow-3]-dao.SelSKUConfigDAOImpl: Retriving selected sku configuration from file /usr/local/dataprotection/var/configmgr/server_data/skuconfig/selskuconfig.xml
2024-01-31 23:39:33,880 INFO  [upgrade-workflow-3]-dao.SelSKUConfigDAOImpl: Successfully retrieved selected sku configuration
2024-01-31 23:39:33,880 INFO  [upgrade-workflow-3]-vi.VCenterService: Input for OVA.OVF deployment VMParameters [fileLocation=/data01/tmp/patch/products/VCSA/binaries/vcsa.ova, fileType=OVA, vmName=DataProtection-VCSA, instanceId=VMware_vCenter_Server_Appliance, skipMfCheck=false, networkPropertyMap={Network 1=DP-appliance-internal}]

  3. When logging into the vCenter over SSH as user "root," running the command ipv4.list shows a missing default gateway for the private interface (labeled as "nic0").

    Command> ipv4.list
Config:
  1:
      Interface: nic0
      Mode: static
      Address: 192.168.100.99
      Prefix: 27
      Defaultgateway: ''
      Updateable: True
  2:
      Interface: nic1
      Mode: static
      Address: 10.1.2.3
      Prefix: 24
      Defaultgateway: 10.1.2.1
      Updateable: True

  4. Running route -n or netstat -rn as from the vCenter CLI bash mode (Accessed by first running "shell" from the vCenter CLI) shows no entry for private NIC 0.

    root@dpappliance-vcsa [ ~ ]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.1.2.1        0.0.0.0         UG    0      0        0 eth1
0.0.0.0         192.168.100.100 0.0.0.0         UG    0      0        0 eth0
10.1.2.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.100.96  0.0.0.0         255.255.255.224 U     0      0        0 eth0

root@dpappliance-vcsa [ ~ ]# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.1.2.1        0.0.0.0         UG        0 0          0 eth1
0.0.0.0         192.168.100.100 0.0.0.0         UG        0 0          0 eth0
10.1.2.0        0.0.0.0         255.255.255.0   U         0 0          0 eth1
192.168.100.96  0.0.0.0         255.255.255.224 U         0 0          0 eth0

     

Cause

The VCSA upgrade failed due to a failure to initialize networking during VCSA deployment and first boot. The gateway value is retrieved from the old VCSA, which may fail if this field is blank or missing. This applies for both NIC0 and NIC1.

 

Resolution

This issue cannot be resolved in the middle of the upgrade process, this needs redeployment, but we can precheck this before upgrading.

 

For Rack Models DP5000 or 8000

  1. Log in to VCSA with root, but do not enter "shell" mode.

  2. Run the command ipv4.list

    Command> ipv4.list
Config:
  1:
      Interface: nic0
      Mode: static
      Address: 192.168.100.99
      Prefix: 27
      Defaultgateway: ''
      Updateable: True
  2:
      Interface: nic1
      Mode: static
      Address: 10.1.2.2
      Prefix: 24
      Defaultgateway: 10.1.2.1
      Updateable: True

    For NIC 0 (Internal) we must set it to 192.168.100.100.
    For NIC 1 (External) we must set it to the customer's gateway, you can also run route -n on ACM to check this gateway.

    Below Example we are taking NIC 0 (Internal) which is missing, and we must set it to 192.168.100.100.

     

    Note: You must go to "shell" mode by typing the shell command. If you are seeing the prompt that says "Command >" you ARE NOT in shell mode. You must be seeing "#" as your prompt.

     

    Command>shell

root@dpappliance-vcsa [ ~ ]# /opt/vmware/share/vami/vami_config_net
 
Main Menu
 
0)      Show Current Configuration (scroll with Shift-PgUp/PgDown)
1)      Exit this program
2)      Default Gateway
3)      Hostname
4)      DNS
5)      Proxy Server
6)      IP Address Allocation for eth0
7)      IP Address Allocation for eth1
Enter a menu number [0]: 2

     

    WARNING: If any of the interfaces for this VM use DHCP, the Hostname, DNS, and Gateway parameters are overwritten by information from the DHCP server.

     

    Type Ctrl-C to go back to the Main Menu.

    0)      eth0
1)      eth1
Choose the interface to associate with the default gateway [0]: 0
Gateway will be associated with eth0.
IPv4 Default Gateway [10.60.9.1]: 192.168.100.100
IPv6 Default Gateway []:
Reconfiguring eth0...
net.ipv6.conf.eth0.disable_ipv6 = 1
Network parameters successfully changed to requested values

  3. Exit shell and check the gateway.

    root@dpappliance-vcsa [ ~ ]# exit
logout
Command> ipv4.list
Config:
  1:
      Interface: nic0
      Mode: static
      Address: 192.168.100.99
      Prefix: 27
      Defaultgateway: 192.168.100.100
      Updateable: True
  2:
      Interface: nic1
      Mode: static
      Address: 10.60.9.50
      Prefix: 24
      Defaultgateway: 10.60.9.1
      Updateable: True
 
Command>

    You can also go to shell and run route -n or netstat -rn.

    root@dpappliance-vcsa [ ~ ]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.60.9.1       0.0.0.0         UG    0      0        0 eth1
0.0.0.0         192.168.100.100 0.0.0.0         UG    0      0        0 eth0
10.60.9.0       0.0.0.0         255.255.255.0   U     0      0        0 eth1
192.168.100.96  0.0.0.0         255.255.255.224 U     0      0        0 eth0

     

    root@dpappliance-vcsa [ ~ ]# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.60.9.1       0.0.0.0         UG        0 0          0 eth1
0.0.0.0         192.168.100.100 0.0.0.0         UG        0 0          0 eth0
10.60.9.0       0.0.0.0         255.255.255.0   U         0 0          0 eth1
192.168.100.96  0.0.0.0         255.255.255.224 U         0 0          0 eth0
root@dpappliance-vcsa [ ~ ]#

 

For DP4400 Models

For DP4400, it only has one Nic as nic0 with public IP, no private IP, and workflow checks public gateway from commonconfig.xml.

 

Note: Only if FIPS a new Nic nic1 gets added to the 4400 with VC private IP.

 

  1. Log in to ACM as root.

  2. Check the commonconfig.xml

    acm-4400-crk-3:~ # cat /usr/local/dataprotection/var/configmgr/server_data/config/commonconfig.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<CommonConfig>
<ipv6Enabled>false</ipv6Enabled>
<fipsEnabled>false</fipsEnabled>
<onInternalNetwork>false</onInternalNetwork>
<uneditable>false</uneditable>
<GeneralSettings>
<timeZone>America/Denver</timeZone>
<ntpServerIP>10.60.9.97</ntpServerIP>
<snmpServerIP></snmpServerIP>
</GeneralSettings>
<ManagementNetworkSettings>
<subnetMask>255.255.255.0</subnetMask>
<gateway>10.60.9.1</gateway>

    If empty you must fix it by adding the proper gateway.

    Command> ipv4.list
Config:
  1:
      Interface: nic0
      Mode: static
      Address: 10.60.9.145
      Prefix: 24
      Defaultgateway: 10.60.9.1
      Updateable: True

Command>

root@vc-4400-crk-3 [ ~ ]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.60.9.1       0.0.0.0         UG    0      0        0 eth0
10.60.9.0       0.0.0.0         255.255.255.0   U     0      0        0 eth0

root@vc-4400-crk-3 [ ~ ]# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         10.60.9.1       0.0.0.0         UG        0 0          0 eth0
10.60.9.0       0.0.0.0         255.255.255.0   U         0 0          0 eth0
root@vc-4400-crk-3 [ ~ ]#

 

Affected Products

PowerProtect DP4400, PowerProtect DP5300, PowerProtect DP5800, PowerProtect DP8300, PowerProtect DP8800, Integrated Data Protection Appliance Family, Integrated Data Protection Appliance Software, PowerProtect DP5900, PowerProtect DP8400 , PowerProtect DP8900 ...
Article Properties
Article Number: 000221888
Article Type: Solution
Last Modified: 27 Mar 2025
Version:  5
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.