透過 NetWorker/NMDA 進行 Sybase 伺服器備份失敗,並出現 I/O 錯誤,最後以 Sybase Sybmultbuf 程序當機而告終

Summary: 透過 NetWorker/NMDA 進行 Sybase 伺服器備份失敗,且發生 I/O 錯誤,最後以 Sybase Sybmultbuf 程序當機而告終。

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.

Symptoms

  • 用於資料庫應用程式的 NetWorker 模組 (NMDA) Sybase 備份失敗
  • nsrdasv 記錄了這些錯誤
24146:nsrdasv: Error from server SL4_BS: Msg 419402, Level 2, State 1
Backup Server: 4.194.2.1: Internal error. The srv_select() function was called to complete an I/O for 2240 bytes. After retrying 2241 times only 0 bytes have been read.24146:nsrdasv: Error from server SL4_BS: Msg 414102, Level 2, State 75
Backup Server: 4.141.2.75: [0] The 'read' call failed for database/archive device while working on stripe device 'nsrsyb::model..3.1./nsr/apps/tmp/sybtmp_1671421.txt::000' with error number 0 (Success). Refer to your operating system documentation for further details.24146:nsrdasv: Error from server SL4_BS: Msg 406203, Level 3, State 1
Backup Server Internal Error: 4.62.3.1: Multibuffering subprocesses died, archive nsrsyb::model..3.1./nsr/apps/tmp/sybtmp_1671421.txt::000.24146:nsrdasv: Error from server SL4_BS: Msg 414102, Level 2, State 66
Backup Server: 4.141.2.66: [0] The 'write' call failed for database/archive device while working on stripe device 'nsrsyb::model..3.1./nsr/apps/tmp/sybtmp_1671421.txt::000' with error number 32 (Broken pipe). Refer to your operating system documentation for further details.24146:nsrdasv: Error from server SL4_BS: Msg 406203, Level 3, State 8
Backup Server Internal Error: 4.62.3.8: Multibuffering subprocesses died, archive nsrsyb::model..3.1./nsr/apps/tmp/sybtmp_1671421.txt::000.24146:nsrdasv: Error from server SL4_BS: Msg 603102, Level 2, State 2
Backup Server: 6.31.2.2: Volume rejected.24146:nsrdasv: Error from server SL4_BS: Msg 101402, Level 2, State 2
Backup Server: 1.14.2.2: Unrecoverable I/O or volume error. This DUMP or LOAD session must exit.24146:nsrdasv: Error from server SL4_BS: Msg 414102, Level 2, State 66
  • 當 NMDA API 使用 OpenSSL 執行加密時,Sybmultbuf 核心:
 A00: SYBMULTBUF: bs_trap; process received signal 11
 A00: 0x7fc08ac64b20 : (killpg+0x40) [0x7fc08ac64b20]
 A00: 0x7fc0898fba79 : (EVP_MD_CTX_clear_flags+0x9) [0x7fc0898fba79]
 A00: 0x7fc0898e9461 : (EVP_DigestInit_ex+0x21) [0x7fc0898e9461]
 A00: 0x7fc08990af31 : (HMAC_Init_ex+0x211) [0x7fc08990af31]
 A00: 0x7fc088a1a219 : (comssl_OpenSSL_hmac_init+0x69) [0x7fc088a1a219]
 A00: 0x7fc0889ab8f9 : (cryptoiface_get_hmac+0x89) [0x7fc0889ab8f9]
 A00: 0x7fc0889ac1c5 : (gsslgtov1_export_sec_context+0x285) [0x7fc0889ac1c5]
 A00: 0x7fc0889af5b0 : (gsslgtov1_init_sec_context+0xe20) [0x7fc0889af5b0]
 A00: 0x7fc0889a1dbc : (authgss_create+0x5dc) [0x7fc0889a1dbc]
 A00: 0x7fc0889af6e0 : (gsslgtov1_init_sec_context+0xf50) [0x7fc0889af6e0]
 A00: 0x7fc0889b0553 : (gsslgtov1_acquire_cred+0xad3) [0x7fc0889b0553]
 A00: 0x7fc08898673c : (lgto_parms+0x127c) [0x7fc08898673c]
 A00: 0x7fc088987909 : (acquire_lg_authgen_cred_lsp_host_override+0x299) [0x7fc088987909]
 A00: 0x7fc088987b87 : (acquire_lg_authgen_cred+0x37) [0x7fc088987b87]
 A00: 0x7fc0889822ba : (rap_bind+0x41a) [0x7fc0889822ba]
 A00: 0x7fc088992f71 : (resdb_net_encrypt+0xd1) [0x7fc088992f71]
 A00: 0x7fc0888789d4 : (nsr_ladb_net+0xc4) [0x7fc0888789d4]
 A00: 0x7fc088878379 : (nsr_findserver+0x109) [0x7fc088878379]
 A00: 0x7fc0887ad5b3 : (_nwbsa_open_options_session+0x243) [0x7fc0887ad5b3]
 A00: 0x7fc08879d145 : (BSAInit+0x275) [0x7fc08879d145]
 A00: 0x7fc08879b273 : (init_bsa+0x33) [0x7fc08879b273]
 A00: 0x7fc088797e1e : (syb_open+0xa8e) [0x7fc088797e1e]
 A00: 0x44d691 : (api_open+0x501) [0x44d691]
 A00: 0x42924f : (ctl_job+0x249) [0x42924f]
 A00: 0x425fe8 : (archive_proc+0x593) [0x425fe8]
 A00: 0x42526a : (sybmbuf_main+0x66e) [0x42526a]
 A00: 0x424bfa : (main+0x1b) [0x424bfa]
 A00: 0x7fc08ac50cf3 : (__libc_start_main+0xf3) [0x7fc08ac50cf3]
 A00: 0x424b09 : (_start+0x29) [0x424b09]
 D00: SYBMULTBUF ERROR: Emulator interprocess communication failed with error state = 9, error code=0, system message=Pipe I/O returned 0 bytes.
 Open Server Error: 16248.10.0: Error in srv_select() - file descriptor 12 is no longer active
 Backup Server: 4.194.2.1: Internal error. The srv_select() function was called to complete an I/O for 2240 bytes. After retrying 2241 times only 0 bytes have been read.
 Backup Server: 4.141.2.75: [0] The 'read' call failed for database/archive device while working on stripe device 'nsrsyb::...1::000' with error number 0 (Success). Refer to your operating system documentation for further details.
 Backup Server Internal Error: 4.62.3.1: Multibuffering subprocesses died, archive nsrsyb::...1::000.
 Backup Server: 4.141.2.66: [0] The 'write' call failed for database/archive device while working on stripe device 'nsrsyb::...1::000' with error number 32 (Broken pipe). Refer to your operating system documentation for further details.
 Backup Server Internal Error: 4.62.3.8: Multibuffering subprocesses died, archive nsrsyb::...1::000.
.....
Jul 6 20:23:36 2022: D02: SYBMULTBUF ERROR: Emulator interprocess communication failed with error state = 9, error code=0, system message=Pipe I/O returned 0 bytes.
Jul 6 20:23:36 2022: Open Server Error: 16248.10.0: Error in srv_select() - file descriptor 17 is no longer active
Jul 6 20:23:36 2022: Open Server Error: 16248.10.0: Error in srv_select() - file descriptor 17 is no longer active

Cause

故障是由於作業系統和 NMDA 的 Openssl libcrypto.so 程式庫之間發生衝突。
Sybase 備份伺服器會載入 Red Hat 8.6 的 libcrypto.so.1.1.1 程式庫,如 ldd 輸出所示。
$ ldd ./backupserver
        linux-vdso.so.1 (0x00007ffe6d6db000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007faea79ab000)
        librt.so.1 => /lib64/librt.so.1 (0x00007faea77a3000)
        libm.so.6 => /lib64/libm.so.6 (0x00007faea7421000)
        libnsl.so.1 => /lib64/libnsl.so.1 (0x00007faea7207000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007faea6fe7000)
        libc.so.6 => /lib64/libc.so.6 (0x00007faea6c22000)
        /lib64/ld-linux-x86-64.so.2 (0x00007faea7baf000)
        libtirpc.so.3 => /lib64/libtirpc.so.3 (0x00007faea69ef000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007faea679a000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007faea64b0000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007faea6299000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007faea6095000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007faea5e84000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007faea5c80000)
        libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007faea5797000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007faea5580000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007faea5356000)
        libz.so.1 => /lib64/libz.so.1 (0x00007faea513e000)
        libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007faea4eba000)
NMDA 使用 LIBCOMMONSSL (openssl 1.0.0) 呼叫 OpenSSL 功能,但由於 Sybase 已載入 1.1.1 程式庫,因此備份會失敗。

Resolution

因應措施:

在 Sybase 備份伺服器執行指令檔中預載 NMDA libcrypto.so.1.0.0 程式庫。重新啟動 Sybase 備份伺服器,
 

  1. 在 Sybase 用戶端上,以 Sybase 使用者身分登入。
  2. 前往 Sybase 安裝主目錄
  3. 來源 Sybase 環境指令檔。
$ . ./SYBASE.sh
  1. 在 RUN 指令檔中匯出 LD_PRELOAD 變數。儲存變更。
$ cd $SYBASE/$SYBASE_ASE/install
$ vi RUN_<SERVER>_BS
# Multibuf executable path:     /home/sybase16sp4/ASE-16_0/bin/sybmultbuf
# Backup Server name:   DELLSP4_BS
#
LD_PRELOAD=/usr/lib/nsr/apps/lib64/libcrypto.so.1.0.0
export LD_PRELOAD
/home/sybase16sp4/ASE-16_0/bin/backupserver \
-e/home/sybase16sp4/ASE-16_0/install/DELLSP4_BS.log \
-N25 \
注意:較新的 NetWorker 版本支援不同版本的 libcrypto 程式庫。請務必檢查 /usr/lib/nsr/apps/lib64 下的 libcrypto 檔案,以確保預先載入正確的程式庫。
  1.  重新啟動 Sybase 備份伺服器。
A. 在 isql 提示符下停止 Sybase 備份伺服器。
$ cd $SYBASE/$SYBASE_OCS/bin
$ Isql -U<user> -P<Password> -S<Sybase Server>
Isql> shutdown SYB_BACKUP
Isql> quit
B. 啟動 Sybase 備份伺服器:
$ cd $SYBASE/$SYBASE_ASE/install
$ startserver -f RUN_<Sybase server>_BS
C. 確認 Sybase 備份伺服器正在執行中:
$ ./showserver

Additional Information

環境:

Red Hat 8.6
SAP ASE 16 SP4 PL02
NetWorker/NMDA 19.5.0.5

Affected Products

NetWorker, NetWorker Module for Databases and Applications

Products

NetWorker Family
Article Properties
Article Number: 000205566
Article Type: Solution
Last Modified: 04 Dec 2024
Version:  2
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.