DB2 backup fails with SQL2071N "/usr/lib/libnsrdb2.so". Reason code: "2"

DB2 backups fail with the following error post upgrade of Database from V10.5 to V11.5

Error:

DB2 backup fails with SQL2071N "/usr/lib/libnsrdb2.so". Reason code: "2"

DB2DIAG logs have the following error.

2025-01-13-20.16.25.864739+060 E79273449E571         LEVEL: Error (OS)
PID     : 29019                TID : 139807381163456 PROC : db2vend (db2logmgr.meth1 - 267 (
INSTANCE: db2prd               NODE : 000
HOSTNAME: labhostPRD-00
FUNCTION: DB2 UDB, oper system services, sqloLoadModule, probe:191
CALLED  : OS, -, dlopen
DATA #1 : Library name or path, 21 bytes
/usr/lib/libnsrdb2.so
DATA #2 : shared library load flags, PD_TYPE_LOAD_FLAGS, 4 bytes
0
DATA #3 : String, 92 bytes
/usr/lib/nsr/apps/lib64/libssl.so.1.1: undefined symbol: EVP_idea_cbc, version OPENSSL_1_1_0
 
2025-01-13-20.16.25.864904+060 I79274021E462         LEVEL: Error
PID     : 29019                TID : 139807381163456 PROC : db2vend (db2logmgr.meth1 - 267 (
INSTANCE: db2prd               NODE : 000
HOSTNAME: labhostPRD-00
FUNCTION: DB2 UDB, database utilities, sqluFencedVendorProcessCB::loadLibrary, probe:104
MESSAGE : ZRC=0x870F009B=-2029059941=SQLO_MOD_LOAD_FAILED
          "Dynamic library load failed."
DATA #1 : String, 21 bytes
/usr/lib/libnsrdb2.so

If OpenSSL version 3 is installed on the system.
db2vend process relies only on OpenSSL version 3, however the libnsrdb2 process will load the OpenSSL version 1 present in NMDA

OS OpenSSL path: /usr/lib64/
NMDA OpenSSL Path: /usr/lib/nsr/apps/lib64/

Below is the snippet of a non-working host.

Two db2vend process

labhostPRD-00:(/root)(root)#ps -ef | grep db2vend | grep -v grep
db2prd   23876 23452  0 Feb05 ?        00:00:00 db2vend (PD Vendor Process - 1) 0
db2prd   24856 23452  0 Feb11 ?        00:00:04 db2vend (db2logmgr.meth1 - 27 (PRD)) 0

First db2vend with PID 23876 loads OpenSSL V3 from OS OpenSSL Path

labhostPRD-00:(/root)(root)#lsof -p 23876 | grep -E "lib(ssl|crypto)"
db2vend 23876 db2prd DEL    REG    8,4           51527190 /usr/lib64/libcrypto.so.3.1.4
db2vend 23876 db2prd DEL    REG    8,4           51527189 /usr/lib64/libcrypto.so.1.1
db2vend 23876 db2prd DEL    REG    8,4           51784225 /usr/lib64/libssl.so.3.1.4

Second db2vend with PID 24856 loads OpenSSL V1 from NMDA OpenSSL Path (/usr/lib/nsr/apps/lib64/libssl.so.1.1) & OS OpenSSL Path

labhostPRD-00:(/root)(root)#lsof -p 24856 | grep -E "lib(ssl|crypto)"
db2vend 24856 db2prd mem    REG    8,4    721456 50354171 /usr/lib/nsr/apps/lib64/libssl.so.1.1
db2vend 24856 db2prd mem    REG    8,4   5715664 50354411 /usr/lib64/libcrypto.so.3.1.4
db2vend 24856 db2prd mem    REG    8,4   3924472 50360290 /usr/lib64/libcrypto.so.1.1
db2vend 24856 db2prd mem    REG    8,4    743784 50354413 /usr/lib64/libssl.so.3.1.4

The above mix-up of OpenSSL version is the cause of the Error.
NMDA OpenSSL version 1 is compiled with "IDEA encryption algorithm for EVP" but OS OpenSSL Version 1 are not.

1. This issue is addressed in NMDA 19.12, NMDA shipped with OpenSSL V3.
2. Uninstalling OpenSSL version 3 would most likely help, because db2vend would be then forced to load OS libraries libssl.so.1.1 and libcrypto.so.1.1, and as a result, initialization of the vendor interface would not need to load libraries distributed with the NMDA
3. Workaround: The DB2 environment variable used as a workaround to force the /usr/lib/nsr/apps/lib64/ path using the follwoing IBM document.  DB2LIBPATH [ibm.com]