NVE upgrade to 19.9 fails with "Reconfiguring NetWorker Management UI Settings Failed"
Summary: During an upgrade of the NetWorker Virtual Edition (NVE) appliance to 19.9.x the upgrade fails at step 43 of 51 stating "Reconfiguring NetWorker Management UI Settings Failed."
Symptoms
- NetWorker Virtual Editions (NVE) upgrade to 19.9.0.2 from an earlier release
- The NVE
/data01/avamar/repo/temp/*/tmp/workflow.logreports:
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: ---------- BEGIN ReconfigureNetworkerUi::do ----------
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: >>> [make_temp_script] 222 very_current_user=root, as_user=root, ssh command is: (systemctl stop nwui) >/usr/local/avamar/var/run_command-sysout.20293.47234543999660 2>&1
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: no output seen from "systemctl stop nwui"
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: "systemctl stop nwui", exit status=0 (success)
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: >>> [make_temp_script] 222 very_current_user=root, as_user=root, ssh command is: (/opt/nwui/scripts/nwui_configure.sh -silent /space/avamar/repo/temp/NveUpgrade-19.9.0-266.avp_1699556217033/tmp/nmc_ui_config.txt) >/usr/local/avamar/var/run_command-sysout.20293.47234543999660 2>&1
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: - - - - - - - BEGIN(output) - - - - - - -
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO:
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: ERROR: Either keystore path or password is incorrect
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: - - - - - - - END
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 ERROR: "/opt/nwui/scripts/nwui_configure.sh -silent /space/avamar/repo/temp/NveUpgrade-19.9.0-266.avp_1699556217033/tmp/nmc_ui_config.txt", exit status=1 (error)
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: post: "/opt/nwui/scripts/nwui_configure.sh -silent /space/avamar/repo/temp/NveUpgrade-19.9.0-266.avp_1699556217033/tmp/nmc_ui_config.txt", exit status=1 (error)
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 ERROR: ERROR: Either keystore path or password is incorrect
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: post: ERROR: Either keystore path or password is incorrect
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: >>> [make_temp_script] 222 very_current_user=root, as_user=, ssh command is: (rpm -q dpnserver --last | head -1) >/usr/local/avamar/var/run_command-sysout.20293.47234543999660 2>&1
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: >>> [make_temp_script] 222 very_current_user=root, as_user=admin, ssh command is: (/usr/local/avamar/bin/avhostname -f) >/usr/local/avamar/var/run_command-sysout.20293.47234543999660 2>/usr/local/avamar/var/run_command-syserr.20293.47234543999660
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: - - - - - - - BEGIN(output) - - - - - - -
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: NVE_HOSTNAME
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: - - - - - - - END
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: "/usr/local/avamar/bin/avhostname -f", exit status=0 (success)
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: Report home message: 2023-12-07 13:22:09 -0500|NVE_HOSTNAME|unknown|NveUpgrade|19.9.0.266|/space/avamar/repo/temp/NveUpgrade-19.9.0-266.avp_1699556217033/tmp|ReconfigureNetworkerUi|Reconfiguring NetWorker Management UI Settings (43 of 52)|ERROR: ReconfigureNetworkerUi::do - operation failed
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 ERROR: - operation failed
YYYY-MM-DD HH:mm:SS (TIMEZONE) 47234543999660 INFO: post: - operation failed
- Manually running the NetWorker Web User Interface (NWUI) configuration script attempted by the NVE upgrade:
/opt/nwui/scripts/nwui_configure.sh -silent /space/avamar/repo/temp/*/tmp/nmc_ui_config.txt
The following error is returned:
keytool error: java.io.IOException: parseAlgParameters failed: PBE parameter parsing error: expecting the object identifier for AES cipher
Cause
This issue was not reproducible in testing.
Resolution
Workaround:
1. Delete the NetWorker Runtime Environment (NRE) "latest" symbolic link:
rm -rf /opt/nre/java/latest
2. Confirm which Java Runtime Environment (JRE) package is installed under /usr/java.
ls -l /usr/java
nve:~ # ls -l /usr/java
total 4
lrwxrwxrwx 1 root root 16 Oct 29 2020 default -> /usr/java/latest
drwxr-xr-x 8 root root 4096 Jun 16 2023 jre1.8.0_341-amd64
lrwxrwxrwx 1 root root 28 Jun 16 2023 latest -> /usr/java/jre1.8.0_341-amd64
3. Re-create the NRE java latest symbolic link using the /usr/java/jre path collected from the above command:
ln -s /usr/java/jre /opt/nre/java/latest
nve:~ # ln -s /usr/java/jre1.8.0_341-amd64 /opt/nre/java/latest nve:~ # ls -l /opt/nre/java/latest lrwxrwxrwx 1 root root 28 Feb 9 13:39 /opt/nre/java/latest -> /usr/java/jre1.8.0_341-amd64
4. Complete the NWUI configuration:
/opt/nwui/scripts/nwui_configure.sh -silent /space/avamar/repo/temp/*/tmp/nmc_ui_config.txt
5. Return to the NVE Installation Manager (Web UI) and complete the upgrade.
6. Once the upgrade completes, reset the NRE symbolic link back to the NRE Java instance:
A. Delete the symbolic link for /opt/nre/java/latest:
rm -rf /opt/nre/java/latest
B. Confirm the /opt/nre version:
ls -l /opt/nre/java
C. Re-create the symbolic link using the NRE Java Runtime Environment (JRE) folder:
ln -s /opt/nre/java/jre#.#.#_### /opt/nre/java/latest