Avamar: Data Domain wird nicht in der Avamar-Web- oder -Administratorkonsole angezeigt

• Data Domain wird nicht in der Avamar-Benutzeroberfläche (UI) oder der Avamar Administratorkonsole angezeigt.
• Durch einen Neustart des Management Console Service (MCS) wird das Problem nicht behoben.

admin@av-server:~/>: mccli dd show-prop --name=ddve01.lab.com
1,30943,Data Domain system does not exist.
admin@av-server:~/>: 

• Neustart über die Befehlszeilenschnittstelle (CLI) mit mcddrsnmp Das Problem wird dadurch nicht behoben.

root@av-server:~/#: mcddrsnmp restart
INFO: Stopping Data Domain SNMP Manager....
INFO: Data Domain SNMP Manager is not running
INFO: Starting Data Domain SNMP Manager....
INFO: Connecting to MCS Server: av-server.emn.com at port: 7778...
INFO: Successfully connected to MCS Server: av-server.lab.com at port: 7778.
INFO: No trap listeners were started, Data Domain SNMP Manager didn't start.
root@av-server:~/#: 

• ddrmaint Der Befehl funktioniert einwandfrei:

admin@av-server:~/>: ddrmaint read-ddr-info --format=full
====================== Read-DDR-Info ======================

 System name        : ddve01.lab.com
 System ID          : 608023E84E6C40CFB5F6608023E84E6123456789
 DDBoost user       : ddboost
 System index       : 1
 Replication        : True
 CP Backup          : True
 Model number       : DD VE
 Serialno           : AUDVHS64123456
 DDOS version       : 7.10.1.20-1090468
 System attached    : 2023-05-26 12:33:29 (1685104409)
 System max streams : 16
 RL Mode            : -1
 RL Status          : -1
admin@av-server:~/>:

• Während des MCS-Starts wird die folgende Meldung in angezeigt /usr/local/avamar/var/mc/server_log/mcserver.log.0:

INFO: Collection of DdrInfo is null or empty.

• Das Skript für die proaktive Integritätsprüfung zeigt, dass die MCS-Leerung 6 Tage lang fehlgeschlagen ist:

ERROR: No MCS flush in past 24 hours. Last one is 2024-08-30 08:03:43
RESOLUTION: Try 'mcserver.sh --flush' to see why MCS flushes are failing
# --> MC flush in past 24 hours      *FAILED*

• List MC_BACKUPS Bestätigt das Problem:

admin@av-server:~/>: avtar --backups --path=/MC_BACKUPS --count=5
avtar Info <5551>: Command Line: /usr/local/avamar/bin/avtar.bin --flagfile=/usr/local/avamar/etc/usersettings.cfg --server=av-server --vardir=/usr/local/avamar/var --bindir=/usr/local/avamar/bin --id=root --password=**************** --vardir=/usr/local/avamar/var --bindir=/usr/local/avamar/bin --sysdir=/usr/local/avamar/etc --backups --account=/MC_BACKUPS --count=5
avtar Info <7977>: Starting at 2024-09-05 22:50:48 IST [avtar Dec  8 2023 07:07:43 19.10.100-135 Linux-x86_64]
avtar Info <6555>: Initializing connection
avtar Info <5552>: Connecting to Avamar Server (av-server)
avtar Info <5554>: Connecting to one node in each datacenter
avtar Info <5583>: Login User: "root", Domain: "default", Account: "/MC_BACKUPS"
avtar Info <5580>: Logging in on connection 0 (server 0)
avtar Info <5582>: Avamar Server login successful
avtar Info <10632>: Using Client-ID='51947b9a14df3111ca78e435008cde1234567890'
avtar Info <5550>: Successfully logged into Avamar Server [19.10.0-135]
avtar Info <19849>: Selecting 5 backups
avtar Info <7377>: Backups for /MC_BACKUPS as of 2024-09-05 22:50:48 IST
    Date      Time    Seq       Label           Size     Plugin    Working directory         Targets
 ---------- -------- ----- ----------------- ---------- -------- --------------------- -------------------
 2024-08-30 08:03:43   904                      565563K Linux    /usr/local/avamar     var/mc/server_data
 2024-08-30 07:46:27   903                      565518K Linux    /usr/local/avamar     var/mc/server_data
 2024-08-29 08:02:17   902                      566738K Linux    /usr/local/avamar     var/mc/server_data
 2024-08-29 07:46:44   901                      566693K Linux    /usr/local/avamar     var/mc/server_data
 2024-08-28 08:02:39   900                      567929K Linux    /usr/local/avamar     var/mc/server_data
avtar Info <5314>: Command completed (exit code 0: success)
admin@av-server:~/>:

• Der Befehl /usr/local/avamar/var/mc/server_log/flush.log zeigt den folgenden Fehler an:

...
2024-09-05 08:01:01 avtar Info <5552>: Connecting to Avamar Server (av-server)
2024-09-05 08:01:01 avtar Info <5554>: Connecting to one node in each datacenter
2024-09-05 08:01:01 avtar Info <5993>: - Connect: Connected to 123.123.1.113:29000, Priv=0, SSL Cipher=ECDHE-RSA-AES256-GCM-SHA384
2024-09-05 08:01:01 avtar Info <5993>: - Datacenter 0 has 1 nodes: Connected to 123.123.1.113:29000, Priv=0, SSL Cipher=ECDHE-RSA-AES256-GCM-SHA384
2024-09-05 08:01:01 avtar Info <5583>: Login User: "root", Domain: "default", Account: "/MC_BACKUPS"
2024-09-05 08:01:01 avtar Info <5580>: Logging in on connection 0 (server 0)
2024-09-05 08:01:01 avtar Error <5126>: Login error 69: Authentication failure  (Invalid password?  User disabled?)
2024-09-05 08:01:01 avtar FATAL <8941>: Fatal server connection problem, aborting initialization. Verify correct server address and login credentials.
2024-09-05 08:01:01 avtar Info <6149>: Error summary: 2 errors: 8941, 5126
2024-09-05 08:01:01 avtar Info <6645>: Not sending wrapup anywhere.
2024-09-05 08:01:01 avtar Info <5314>: Command failed (2 errors, exit code 10008: cannot establish connection with server (possible network or DNS failure))
--------------------------------------------------------------------------------------------------------
----- END avtar log 2024-09-05 08:01:01 IST  (0 warnings, 2 errors, 1 fatal error)
--------------------------------------------------------------------------------------------------------

• Die Change-Passwords wurden vor sechs Tagen ausgeführt:

root@av-server:/home/admin/#: history | grep change
    1  2023-06-01 19:55:53 change-passwords
  191  2024-08-30 19:42:59 change-passwords
  555  2024-09-05 22:56:13 history | grep change
root@av-server:/home/admin/#:

• Das Kennwort in /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml zeigt an, dass Root und MCUser dasselbe Kennwort verwenden: 

root@av-server:/home/admin/proactive_check/#: grep AP /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
                <entry key="backuprestoreAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="backuponlyAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="rootAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
                <entry key="MCUSERAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
                <entry key="restoreonlyAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="viewuserAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
              <entry key="monarch_license_feature_codes" value="POWER_PROTECT_CAPACITY" />
              <entry key="rabbitmqAP" value="{AES}UhnKR3q+4LSjgqTF5SSquw==" />
root@av-server:/home/admin/proactive_check/#:

• Melden Sie sich jedoch mit MCUser und root bei der Avamar-Benutzeroberfläche an. Das Kennwort ist ein anderes.
• Auf GSAN-Ebene sind die Kennwörter für root und MCUser korrekt:

admin@av-server:~/>: avmgr logn --id=root --ap=*******
1  Request succeeded
-1  privilege level  (enabled,create,read,backup,access,move,delete,maint,manage,fullmanage,noticketrequired,superuser,ignoreacls,readdir,mclogin,opt1,opt2)
2  block type  (directory)
admin@av-server:~/>:

admin@av-server:~/>: avmgr logn --id=MCUser --ap=xxxxxxx
1  Request succeeded
7161  privilege level  (enabled,create,read,backup,access,move,delete,maint,fullmanage,noticketrequired)
2  block type  (directory)
admin@av-server:~/>:

Um dieses Problem zu beheben, verschlüsseln Sie das Kennwort des Root-Servers im mcserver.xml und starten Sie den MCS neu. Dadurch wird der Data Domain SNMP Manager gestartet, wodurch die Data Domain sowohl in der Avamar-Benutzeroberfläche als auch in MCCLI sichtbar wird.

• Setzen Sie das Root-Server-Kennwort im Klartext auf /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml und verschlüsseln Sie erneut.
• Backup-mcserver.xml:

cp -p /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml /usr/local/avamar/var/mc/server_data/prefs/x-mcserver.xml

• Bearbeiten Sie die /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml und geben Sie das Passwort des Root-Servers als Klartext in den rootAP-Wert ein:

admin@av-server:/home/admin/#: vi /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
                <entry key="backuprestoreAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="backuponlyAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="rootAP" value="change-password-here" />
                <entry key="MCUSERAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
                <entry key="restoreonlyAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="viewuserAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
              <entry key="monarch_license_feature_codes" value="POWER_PROTECT_CAPACITY" />
              <entry key="rabbitmqAP" value="{AES}UhnKR3q+4LSjgqTF5SSquw==" />
admin@av-server:/home/admin/#:

• Verschlüsseln Sie das Kennwort:

root@av-server:/home/admin/#: mccipher encrypt --all
**********************************************************************************
* EMC Avamar Management Console (MC).                                            *
* MC Security Tool for Secret Key generation, encryption, decryption and digest. *
**********************************************************************************
Ignore to encrypted preference, since value is NULL or EMPTY: mcserver:/usr/local/avamar/lib:com/avamar/mc/cac/ldap_login_ap
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/lib:com/avamar/mc/cac/ldap_login_ap, will not be stored
Ignore to encrypted preference, since value is NULL or EMPTY: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/cac/ldap_login_ap
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/cac/ldap_login_ap, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/MCUSERAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/backuponlyAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/backuprestoreAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/restoreonlyAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/viewuserAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/msg_broker/rabbitmqAP, will not be stored
WARNING: Can not encrypt some or all MCS Preferences.
All MC DB Tables have been encrypted successfully.
All MCCLI config files have been encrypted successfully.
See MCCipher log for details.
? 2012 EMC Corporation. All rights reserved.
root@av-server:/home/admin#:

• Starten Sie MCS neu. Während des Neustarts wird der Data Domain SNMP Manager gestartet:

INFO: Starting Data Domain SNMP Manager....
INFO: Connecting to MCS Server: av-server.emn.com at port: 7778...
INFO: Successfully connected to MCS Server: av-server.lab.com at port: 7778.
INFO: Trap listeners status:
INFO: Listening to port 163 for traps from [ddve01.lab.com]
INFO: Data Domain SNMP Manager started.

• Die MCS-Spülung funktioniert einwandfrei.
• Data Domain wird erneut in der Avamar-Benutzeroberfläche und MCCLI angezeigt:

admin@av-server:~/>: mccli dd show-prop --name=ddve01.lab.com
0,23000,CLI command completed successfully.
Attribute                                     Value
--------------------------------------------- ---------------------------------
IPv4 Hostname                                 ddve01.lab.com
IPv6 Hostname                                 N/A
Total Capacity (post-comp size)               3038.2 GiB
Server Utilization (post-comp use%)           38%
Bytes Protected (client pre-comp size)        1.1 TB
File System Available (post-comp avail)       1879.3 GiB
File System Used (post-comp used)             1158.9 GiB
User Name                                     ddboost
Default Replication Storage System            Yes
Target For Avamar Checkpoint Backups          Yes
Maximum Streams For Avamar Checkpoint Backups 1
Maximum Streams                               16
Maximum Streams Limit                         20
Instant Access Limit                          1
DDOS Version                                  7.10.1.20-1090468
Serial Number                                 AUDVHS64123456
Model Number                                  DD VE
Encryption Strength                           none
Authentication Mode                           none
Monitoring Status                             SNMP trap manager is not running.
Cloud Tier                                    disabled
DDBoost Licensed                              true
DDBoost Enabled                               true
DDBoost User Enabled                          true
DDBoost User Status                           User Valid
DDBoost Option Status                         Option Enabled
SNMP status                                   Enabled
File System Status                            File System Running
Synchronization of maintenance operations     on
admin@av-server:~/>: