Avamar: Data Domain vises ikke i Avamar Web eller Administrator Console

• Data Domain vises ikke i Avamar User Interface (UI) eller Avamar Administrator-konsollen.
• Start Management Console Service (MCS) på nytt løser ikke problemet.

admin@av-server:~/>: mccli dd show-prop --name=ddve01.lab.com
1,30943,Data Domain system does not exist.
admin@av-server:~/>: 

• Starter på nytt med kommandolinjegrensesnittet (CLI) ved hjelp av mcddrsnmp løser ikke problemet.

root@av-server:~/#: mcddrsnmp restart
INFO: Stopping Data Domain SNMP Manager....
INFO: Data Domain SNMP Manager is not running
INFO: Starting Data Domain SNMP Manager....
INFO: Connecting to MCS Server: av-server.emn.com at port: 7778...
INFO: Successfully connected to MCS Server: av-server.lab.com at port: 7778.
INFO: No trap listeners were started, Data Domain SNMP Manager didn't start.
root@av-server:~/#: 

• ddrmaint Kommandoen fungerer fint:

admin@av-server:~/>: ddrmaint read-ddr-info --format=full
====================== Read-DDR-Info ======================

 System name        : ddve01.lab.com
 System ID          : 608023E84E6C40CFB5F6608023E84E6123456789
 DDBoost user       : ddboost
 System index       : 1
 Replication        : True
 CP Backup          : True
 Model number       : DD VE
 Serialno           : AUDVHS64123456
 DDOS version       : 7.10.1.20-1090468
 System attached    : 2023-05-26 12:33:29 (1685104409)
 System max streams : 16
 RL Mode            : -1
 RL Status          : -1
admin@av-server:~/>:

• Under MCS-start vises følgende melding i /usr/local/avamar/var/mc/server_log/mcserver.log.0:

INFO: Collection of DdrInfo is null or empty.

• Proaktivt helsesjekkskript viser at MCS-spyling sviktet i 6 dager:

ERROR: No MCS flush in past 24 hours. Last one is 2024-08-30 08:03:43
RESOLUTION: Try 'mcserver.sh --flush' to see why MCS flushes are failing
# --> MC flush in past 24 hours      *FAILED*

• List MC_BACKUPS bekrefter problemet:

admin@av-server:~/>: avtar --backups --path=/MC_BACKUPS --count=5
avtar Info <5551>: Command Line: /usr/local/avamar/bin/avtar.bin --flagfile=/usr/local/avamar/etc/usersettings.cfg --server=av-server --vardir=/usr/local/avamar/var --bindir=/usr/local/avamar/bin --id=root --password=**************** --vardir=/usr/local/avamar/var --bindir=/usr/local/avamar/bin --sysdir=/usr/local/avamar/etc --backups --account=/MC_BACKUPS --count=5
avtar Info <7977>: Starting at 2024-09-05 22:50:48 IST [avtar Dec  8 2023 07:07:43 19.10.100-135 Linux-x86_64]
avtar Info <6555>: Initializing connection
avtar Info <5552>: Connecting to Avamar Server (av-server)
avtar Info <5554>: Connecting to one node in each datacenter
avtar Info <5583>: Login User: "root", Domain: "default", Account: "/MC_BACKUPS"
avtar Info <5580>: Logging in on connection 0 (server 0)
avtar Info <5582>: Avamar Server login successful
avtar Info <10632>: Using Client-ID='51947b9a14df3111ca78e435008cde1234567890'
avtar Info <5550>: Successfully logged into Avamar Server [19.10.0-135]
avtar Info <19849>: Selecting 5 backups
avtar Info <7377>: Backups for /MC_BACKUPS as of 2024-09-05 22:50:48 IST
    Date      Time    Seq       Label           Size     Plugin    Working directory         Targets
 ---------- -------- ----- ----------------- ---------- -------- --------------------- -------------------
 2024-08-30 08:03:43   904                      565563K Linux    /usr/local/avamar     var/mc/server_data
 2024-08-30 07:46:27   903                      565518K Linux    /usr/local/avamar     var/mc/server_data
 2024-08-29 08:02:17   902                      566738K Linux    /usr/local/avamar     var/mc/server_data
 2024-08-29 07:46:44   901                      566693K Linux    /usr/local/avamar     var/mc/server_data
 2024-08-28 08:02:39   900                      567929K Linux    /usr/local/avamar     var/mc/server_data
avtar Info <5314>: Command completed (exit code 0: success)
admin@av-server:~/>:

• Informasjonen i /usr/local/avamar/var/mc/server_log/flush.log Viser følgende feil:

...
2024-09-05 08:01:01 avtar Info <5552>: Connecting to Avamar Server (av-server)
2024-09-05 08:01:01 avtar Info <5554>: Connecting to one node in each datacenter
2024-09-05 08:01:01 avtar Info <5993>: - Connect: Connected to 123.123.1.113:29000, Priv=0, SSL Cipher=ECDHE-RSA-AES256-GCM-SHA384
2024-09-05 08:01:01 avtar Info <5993>: - Datacenter 0 has 1 nodes: Connected to 123.123.1.113:29000, Priv=0, SSL Cipher=ECDHE-RSA-AES256-GCM-SHA384
2024-09-05 08:01:01 avtar Info <5583>: Login User: "root", Domain: "default", Account: "/MC_BACKUPS"
2024-09-05 08:01:01 avtar Info <5580>: Logging in on connection 0 (server 0)
2024-09-05 08:01:01 avtar Error <5126>: Login error 69: Authentication failure  (Invalid password?  User disabled?)
2024-09-05 08:01:01 avtar FATAL <8941>: Fatal server connection problem, aborting initialization. Verify correct server address and login credentials.
2024-09-05 08:01:01 avtar Info <6149>: Error summary: 2 errors: 8941, 5126
2024-09-05 08:01:01 avtar Info <6645>: Not sending wrapup anywhere.
2024-09-05 08:01:01 avtar Info <5314>: Command failed (2 errors, exit code 10008: cannot establish connection with server (possible network or DNS failure))
--------------------------------------------------------------------------------------------------------
----- END avtar log 2024-09-05 08:01:01 IST  (0 warnings, 2 errors, 1 fatal error)
--------------------------------------------------------------------------------------------------------

• Endringspassordene løp for seks dager siden:

root@av-server:/home/admin/#: history | grep change
    1  2023-06-01 19:55:53 change-passwords
  191  2024-08-30 19:42:59 change-passwords
  555  2024-09-05 22:56:13 history | grep change
root@av-server:/home/admin/#:

• Passordet i /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml viser roten og MCUser med samme passord: 

root@av-server:/home/admin/proactive_check/#: grep AP /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
                <entry key="backuprestoreAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="backuponlyAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="rootAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
                <entry key="MCUSERAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
                <entry key="restoreonlyAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="viewuserAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
              <entry key="monarch_license_feature_codes" value="POWER_PROTECT_CAPACITY" />
              <entry key="rabbitmqAP" value="{AES}UhnKR3q+4LSjgqTF5SSquw==" />
root@av-server:/home/admin/proactive_check/#:

• Logg på Avamar-grensesnittet med MCUser og rot, passordet er annerledes.
• På GSAN-nivå er passordet for root og MCUser riktig:

admin@av-server:~/>: avmgr logn --id=root --ap=*******
1  Request succeeded
-1  privilege level  (enabled,create,read,backup,access,move,delete,maint,manage,fullmanage,noticketrequired,superuser,ignoreacls,readdir,mclogin,opt1,opt2)
2  block type  (directory)
admin@av-server:~/>:

admin@av-server:~/>: avmgr logn --id=MCUser --ap=xxxxxxx
1  Request succeeded
7161  privilege level  (enabled,create,read,backup,access,move,delete,maint,fullmanage,noticketrequired)
2  block type  (directory)
admin@av-server:~/>:

Du kan løse dette problemet ved å kryptere rotserverpassordet i mcserver.xml og start MCS på nytt. Dette starter Data Domain SNMP Manager, noe som gjør Data Domain synlig i både Avamar-brukergrensesnittet og MCCLI.

• Sett rotserverpassordet i ren tekst på /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml og kryptere igjen.
• Sikkerhetskopiering mcserver.xml:

cp -p /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml /usr/local/avamar/var/mc/server_data/prefs/x-mcserver.xml

• Rediger ikonet /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml og sette rotserverpassordet som ren tekst i rootAP-verdien:

admin@av-server:/home/admin/#: vi /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
                <entry key="backuprestoreAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="backuponlyAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="rootAP" value="change-password-here" />
                <entry key="MCUSERAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
                <entry key="restoreonlyAP" value="{AES}hMKAX9yMuHxbyvltADTR0NALyFSXDMMTCQIqYkry5Bk=" />
                <entry key="viewuserAP" value="{AES}cBqSwCN8Grcb/f735SS9Yw==" />
              <entry key="monarch_license_feature_codes" value="POWER_PROTECT_CAPACITY" />
              <entry key="rabbitmqAP" value="{AES}UhnKR3q+4LSjgqTF5SSquw==" />
admin@av-server:/home/admin/#:

• Krypter passordet:

root@av-server:/home/admin/#: mccipher encrypt --all
**********************************************************************************
* EMC Avamar Management Console (MC).                                            *
* MC Security Tool for Secret Key generation, encryption, decryption and digest. *
**********************************************************************************
Ignore to encrypted preference, since value is NULL or EMPTY: mcserver:/usr/local/avamar/lib:com/avamar/mc/cac/ldap_login_ap
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/lib:com/avamar/mc/cac/ldap_login_ap, will not be stored
Ignore to encrypted preference, since value is NULL or EMPTY: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/cac/ldap_login_ap
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/cac/ldap_login_ap, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/MCUSERAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/backuponlyAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/backuprestoreAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/restoreonlyAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/dpn/users/viewuserAP, will not be stored
encrypted processedValue is null, with prefKey: mcserver:/usr/local/avamar/var/mc/server_data/prefs:com/avamar/mc/msg_broker/rabbitmqAP, will not be stored
WARNING: Can not encrypt some or all MCS Preferences.
All MC DB Tables have been encrypted successfully.
All MCCLI config files have been encrypted successfully.
See MCCipher log for details.
? 2012 EMC Corporation. All rights reserved.
root@av-server:/home/admin#:

• Start MCS på nytt. Under omstarten starter Data Domain SNMP Manager:

INFO: Starting Data Domain SNMP Manager....
INFO: Connecting to MCS Server: av-server.emn.com at port: 7778...
INFO: Successfully connected to MCS Server: av-server.lab.com at port: 7778.
INFO: Trap listeners status:
INFO: Listening to port 163 for traps from [ddve01.lab.com]
INFO: Data Domain SNMP Manager started.

• MCS flush fungerer fint.
• Data Domain vises på nytt i Avamar UI og MCCLI:

admin@av-server:~/>: mccli dd show-prop --name=ddve01.lab.com
0,23000,CLI command completed successfully.
Attribute                                     Value
--------------------------------------------- ---------------------------------
IPv4 Hostname                                 ddve01.lab.com
IPv6 Hostname                                 N/A
Total Capacity (post-comp size)               3038.2 GiB
Server Utilization (post-comp use%)           38%
Bytes Protected (client pre-comp size)        1.1 TB
File System Available (post-comp avail)       1879.3 GiB
File System Used (post-comp used)             1158.9 GiB
User Name                                     ddboost
Default Replication Storage System            Yes
Target For Avamar Checkpoint Backups          Yes
Maximum Streams For Avamar Checkpoint Backups 1
Maximum Streams                               16
Maximum Streams Limit                         20
Instant Access Limit                          1
DDOS Version                                  7.10.1.20-1090468
Serial Number                                 AUDVHS64123456
Model Number                                  DD VE
Encryption Strength                           none
Authentication Mode                           none
Monitoring Status                             SNMP trap manager is not running.
Cloud Tier                                    disabled
DDBoost Licensed                              true
DDBoost Enabled                               true
DDBoost User Enabled                          true
DDBoost User Status                           User Valid
DDBoost Option Status                         Option Enabled
SNMP status                                   Enabled
File System Status                            File System Running
Synchronization of maintenance operations     on
admin@av-server:~/>: