Avamar - NDMP always performs full backup to DD.
Summary: The Network Data Management Protocol (NDMP) backup always runs as a full backup.
Symptoms
The NDMP backup always runs a full backup.
When performing incremental backup of an NDMP client, it always performs a level zero backup.
A variation of the below message is logged to the avndmp log:
avndmp Info <0000>: [snapup-/NAS15] No previous suitable backup found, will perform level zero backup.
Cause
This is caused because the client side has not refreshed the Data Domain (DD) credentials.
The avtar cannot retrieve logs from DD:
avndmp Info <9552>: [snapup-/NAS15] Spawning '/usr/local/avamar/bin/avtar' (avtar --labelnum="164" --showlog --ddr --ddr-index="1" --streamformat-in="cdsf" workresults.xml --vardir="/usr/local/avamar/var/nas05" --bindir="/usr/local/avamar/bin" --sysdir="/usr/local/avamar/etc" --acnt="/NDMP-NAS05/nas05" --id="backuponly" --ap="backuponly1" --server="avamar_lab" --ddr --ddr-index="1" --sessionticket="@xxx" --worktickethash="80fd00edc848c7f7cc83c2f3899b16fb53ef8777" --encrypt="proprietary" --encrypt-strength="cleartext") avndmp Info <6444>: [snapup-/NAS15] Plugin exited with 'code 170: completed with errors, client log should be examined'
And this issue can be tested with the "avtar --showlog" command.
For example:
admin@avamar_lab:~/#: avtar --labelnum="157" --showlog --ddr --ddr-index="1" --streamformat-in="cdsf" workresults.xml --acnt="/NDMP-NAS05/nas05" --id="MCUser" --ap="****" --encrypt="proprietary" --encrypt-strength="cleartext" --server=x.x.x.x
The log indicates a problem logging into the DDR server.
Common messages as seen in the avtar log documenting the issue can be seen below:
avtar Info <7977>: Starting at 2015-04-11 20:31:57 MST [avtar Feb 25 2015 17:22:03 7.1.101-145 Linux-x86_64] avtar Warning <18125>: Calling DDR_OPEN returned result code:5045 message:Function not supported avtar Error <10542>: Data Domain server "avamar_dd2_lab" open failed DDR result code: 5045, desc: Function not supported avtar Error <10512>: Problem logging into the DDR server index:1 avtar Warning <19046>: Unable to get ddr_files.xml for clientid d472726c80d7fdfb548b4c74e79ff065563c7138 and backupid 0 avtar Warning <19047>: Unable to retrieve DDR info for '_2FEMC-NAS15_5FHome3' avtar Warning <19046>: Unable to get ddr_files.xml for clientid d472726c80d7fdfb548b4c74e79ff065563c7138 and backupid 0 avtar Warning <19047>: Unable to retrieve DDR info for '_2FEMC-NAS15_5FHome6' avtar Warning <19046>: Unable to get ddr_files.xml for clientid d472726c80d7fdfb548b4c74e79ff065563c7138 and backupid 0 avtar Warning <19047>: Unable to retrieve DDR info for '_2FEMC-NAS15_5FHome4' avtar Warning <19046>: Unable to get ddr_files.xml for clientid d472726c80d7fdfb548b4c74e79ff065563c7138 and backupid 0
This issue can be confirmed by running the following command:
ddrmaint read-ddr-info
The result shows that the DDOS-version cannot be retrieved.
admin@Avamar-lab:~/#: ddrmaint read-ddr-info
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<avamar dpnid="1385768983" type="default" version="5">
<datadomain count="1">
<ddrconfig client-map-default="true" dd-cert-chain="" ddos-version="" ddrcreatetime="1417399228" ddrid="F4C19BB1A0CC798EAEE62CF0455F0CF76339B109" gsan-backup-target-default="true" hostname="DD860-lab.labdomain" index="1" instant-access-limit="1" ipv4-hostname="DD860-shanghai.localdomain" ipv6-hostname="" max-streams="50" max-streams-for-cp-backup="2" modelno="DD860" password="/vZFomqRdldQwCPHzqXqOQ==" serialno="1FA0736154" token="AQCWfJ9HiJc249eDhLnH8Rc5BPjAMW2/uJQ3MuuYvgrbIw==" username="sysadmin">
<snmp community="x2kxVzW6NyWnVnJrmd+CXA==">
<ports getter-setter="161" trap="163">
</ports>
</snmp>
<client-map>
<client name="/REPLICATE/gen4s-util.sh.lab.emc.com">
</client>
</client-map>
</ddrconfig>
</datadomain>
</avamar>
Resolution
It can see that the full backup can be sent to DD correctly.
This indicates that the credential from Avamar to DD is good.
The DD information must be refreshed without any changes made.
Open the Avamar UI:
Server -> Server Management -> select the issue DD -> click Edit DD domain system -> OK
Then it is OK, as displayed above.