Avamar: Data Domain Integration: Cloud Tier - Marked backups are not going to cloud due to unconfigured schedule

Summary: The backups were marked but no files were found in the cloud.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.

Symptoms

On an Avamar grid integrated with Data Domain, after Cloud Tiering is configured the backups are marked, but remain on the Data Domain while the Cloud location stays empty.

Backups are marked but show no indication of being moved to the Cloud. 
 

Avamar:

admin@AVE01:~/#: mccli backup show --name=/clients/10.0.10.10/VirtualMachines/VMTEST01 
0,23000,CLI command completed successfully.
Created             LabelNum Size    Retention Hostname         Location Consistent Level Tier
------------------- -------- ------- --------- ---------------- -------- ---------------- ------
2019-11-17 06:08 PM 130      86.4 GB DW        AVE01.test.local Local    app              Active
2019-11-16 06:08 PM 129      86.4 GB D         AVE01.test.local Local    app              Active
2019-11-15 06:08 PM 128      86.4 GB D         AVE01.test.local Local    app              Active
--------------- snip ---------------
2019-10-23 07:08 PM 105      86.4 GB D         AVE01.test.local Local    app              Active
2019-10-22 07:07 PM 104      86.4 GB D         AVE01.test.local Local    app              Active
2019-10-21 07:07 PM 103      86.4 GB DW        AVE01.test.local Local    app              Active
2019-10-17 07:08 PM 102      86.4 GB D         AVE01.test.local Local    app              Marked
2019-10-16 07:07 PM 101      86.4 GB D         AVE01.test.local Local    app              Marked
2019-10-15 07:09 PM 100      86.4 GB D         AVE01.test.local Local    app              Marked
--------------- snip ---------------
2019-08-22 07:07 PM 50       86.4 GB D         AVE01.test.local Local    app              Marked
2019-08-21 07:06 PM 49       86.4 GB D         AVE01.test.local Local    app              Marked
2019-08-20 07:08 PM 48       86.4 GB D         AVE01.test.local Local    app              Marked
 

Data Domain:

The "data-movement show history" command shows that the latest activity happened some time ago:

sysadmin@dd3300# data-movement show history
       Start Time            End Time   Bytes Moved   Bytes Moved   Files Moved      Elapsed Time   State
                                         (Pre-comp)   (Post-comp)                      (hh:mm:ss)
-----------------   -----------------   -----------   -----------   -----------   ---------------   ---------
Sep 24 2019 16:55   Sep 24 2019 16:55             0             0             0   0 days 00:00:00   Completed
Sep 24 2019 16:53   Sep 24 2019 16:53             0             0             0   0 days 00:00:00   Completed
-----------------   -----------------   -----------   -----------   -----------   ---------------   ---------

Cause

The data-movement schedule on Data Domain was not configured, so the data did not get migrated to the cloud unit.
 

This can be confirmed by running the following commands on the Data Domain:

sysadmin@dd3300# data-movement schedule show
Data-movement is not scheduled.
sysadmin@dd3300# data-movement status
Data-movement to cloud tier:
----------------------------
Data-movement was started on Sep 24 2019 16:55 and completed on Sep 24 2019 16:55
Moved (post-comp): None, (pre-comp): None,
Files inspected: 0, Files eligible: 0, Files moved: 0, Files failed: 0

Data-movement recall:
---------------------
No recall operations found.
Use 'path all' option to list all recall jobs (up to the maximum supported limit).


 

Resolution

Configure the data-movement schedule.

1. Set the schedule: (In this example, it is set up to run every day at 8PM):

sysadmin@dd3300# data-movement schedule set to-tier cloud days sun-sat time 20:00
Data-movement schedule has been set.
Data-movement is scheduled to run on day(s) "sun,mon,tue,wed,thu,fri,sat" at "20:00" hrs. 
 
 

2. If the customer allows, start the data movement (otherwise wait for the schedule):

sysadmin@dd3300# data-movement start
Data-movement started.
Run "data-movement watch" to monitor progress.

 

As stated in the output above, If started manually, the "data-movement watch" command can be run to monitor progress. The commands below provide additional information.

 

After a few minutes, check the status:

sysadmin@dd3300# data-movement status
data-movement status
Data-movement to cloud tier:
----------------------------
Data-movement:
    3% complete; Elapsed time:  0:01:31
Moved (post-comp): 308.88 KiB, (pre-comp): 717.19 MiB,
Files inspected: 251, Files eligible: 23, Files moved: 0, Files failed: 0

Data-movement recall:
---------------------
No recall operations found.
Use 'path all' option to list all recall jobs (up to the maximum supported limit).


The following command provides more details:

sysadmin@dd3300# data-movement status to-tier cloud detailed
Data-movement to cloud tier:
----------------------------
Data-movement:
    3% complete; Elapsed time:  0:25:42
Moved (post-comp): 9.06 MiB, (pre-comp): 3.54 GiB,
Files inspected: 253, Files eligible: 25, Files moved: 0, Files failed: 0

Data-movement status for MTrees:
--------------------------------
MTree                               Files      Files   Files    Files   Bytes Moved   Bytes Moved   Destination
                                Inspected   Eligible   Moved   Failed    (Pre-comp)   (Post-comp)   Cloud Unit
-----------------------------   ---------   --------   -----   ------   -----------   -----------   -----------
/data/col1/avamar-1562000000*         253         25       0        0     86.42 GiB      9.06 MiB   Avamar
-----------------------------   ---------   --------   -----   ------   -----------   -----------   -----------
(*) Data-movement is in progress for the marked MTrees.

Files currently being moved:
----------------------------
Path Name                                                                                                                                               File Size       Logical   Destination   Elapsed Time
                                                                                                                                                                    Bytes Moved   Cloud Unit        hh:mm:ss
-----------------------------------------------------------------------------------------------------------------------------------------------------   ---------   -----------   -----------   ------------
/data/col1/avamar-1562000000/cur/DELETED/1574121600/1e428c0110b9162726f5217463a247c4728985d8/1D556114FA39B9C/251A33C32514398ED2325F2F5B304D4A9EBC936A    1.61 KiB      583.00 B   Avamar            00:04:36
/data/col1/avamar-1562000000/cur/DELETED/1574121600/1e428c0110b9162726f5217463a247c4728985d8/1D556114FA39B9C/B6FE71BED477868E7EE213BD9E0FAEFEDE55468B    6.39 GiB    574.52 MiB   Avamar            00:04:36
/data/col1/avamar-1562000000/cur/DELETED/1574121600/1e428c0110b9162726f5217463a247c4728985d8/1D556114FA39B9C/D56D73E317F1F82912DE4F4B8364E2AC353F6666   80.01 GiB      3.02 GiB   Avamar            00:24:26
/data/col1/avamar-1562000000/cur/DELETED/1574121600/1e428c0110b9162726f5217463a247c4728985d8/1D556114FA39B9C/7605BFA0D16EA64711A0D54A5145A7B4DB5A1CE1    2.43 KiB      1.39 KiB   Avamar            00:15:07
-----------------------------------------------------------------------------------------------------------------------------------------------------   ---------   -----------   -----------   ------------
 
 

3. After some hours, or any time after a scheduled data movement policy, recheck the activity status:

sysadmin@dd3300# data-movement status
Data-movement to cloud tier:
----------------------------
Data-movement:
    3% complete; Elapsed time:  1:17:06
Moved (post-comp): 813.55 MiB, (pre-comp): 13.84 GiB,
Files inspected: 259, Files eligible: 31, Files moved: 11, Files failed: 0

Data-movement recall:
---------------------
No recall operations found.
Use 'path all' option to list all recall jobs (up to the maximum supported limit).

 

The following command provides more details:

sysadmin@dd3300# data-movement status to-tier cloud detailed
Data-movement to cloud tier:
----------------------------
Data-movement:
    3% complete; Elapsed time:  1:17:16
Moved (post-comp): 813.55 MiB, (pre-comp): 13.84 GiB,
Files inspected: 259, Files eligible: 31, Files moved: 11, Files failed: 0

Data-movement status for MTrees:
--------------------------------
MTree                               Files      Files   Files    Files   Bytes Moved   Bytes Moved   Destination
                                Inspected   Eligible   Moved   Failed    (Pre-comp)   (Post-comp)   Cloud Unit
-----------------------------   ---------   --------   -----   ------   -----------   -----------   -----------
/data/col1/avamar-1562000000*         262         34      14        0     86.42 GiB    813.55 MiB   Avamar
-----------------------------   ---------   --------   -----   ------   -----------   -----------   -----------
(*) Data-movement is in progress for the marked MTrees.

Files currently being moved:
----------------------------
Path Name                                                                                                                                               File Size             Logical   Destination   Elapsed Time
                                                                                                                                                                          Bytes Moved   Cloud Unit        hh:mm:ss
-----------------------------------------------------------------------------------------------------------------------------------------------------   ---------         -----------   -----------   ------------
/data/col1/avamar-1562000000/cur/DELETED/1574208000/1e428c0110b9162726f5217463a247c4728985d8/1D556DA640A1786/A88DCF1006786F4B2EB9BC5B733345CDAD991683    3.24 KiB            2.19 KiB   Avamar            00:14:51
/data/col1/avamar-1562000000/cur/DELETED/1574208000/1e428c0110b9162726f5217463a247c4728985d8/1D556DA640A1786/4E6B85E1E62C6E62F5BCFB969EB82E8E7F8FCE29    1.73 KiB            702.00 B   Avamar            00:05:50
/data/col1/avamar-1562000000/cur/DELETED/1574121600/1e428c0110b9162726f5217463a247c4728985d8/1D556114FA39B9C/D56D73E317F1F82912DE4F4B8364E2AC353F6666   80.01 GiB            7.43 GiB   Avamar            01:15:55
/data/col1/avamar-1562000000/cur/DELETED/1574208000/1e428c0110b9162726f5217463a247c4728985d8/1D556DA640A1786/B410BEB61B20C68EF1048BBF5454FD5877CB23EF   10.50 KiB            9.45 KiB   Avamar            00:05:04
-----------------------------------------------------------------------------------------------------------------------------------------------------   ---------         -----------   -----------   ------------
 
 

4. After the Avamar Garbage Collection has completed (likely the next day as part of the maintenance schedule), confirm that the backups are now tiered:

admin@AVE01:~/#: mccli backup show --name=/clients/10.0.10.10/VirtualMachines/VMTEST01 
0,23000,CLI command completed successfully.
Created             LabelNum Size    Retention Hostname         Location Consistent Level Tier
------------------- -------- ------- --------- ---------------- -------- ---------------- ------
2019-11-20 06:08 PM 133      86.4 GB D         AVE01.test.local Local    app              Active
2019-11-19 06:07 PM 132      86.4 GB D         AVE01.test.local Local    app              Active
2019-11-18 06:07 PM 131      86.4 GB D         AVE01.test.local Local    app              Active
--------------- snip ---------------
2019-10-24 07:08 PM 106      86.4 GB D         AVE01.test.local Local    app              Active
2019-10-23 07:08 PM 105      86.4 GB D         AVE01.test.local Local    app              Active
2019-10-22 07:07 PM 104      86.4 GB D         AVE01.test.local Local    app              Active
2019-10-21 07:07 PM 103      86.4 GB DW        AVE01.test.local Local    app              Marked
2019-10-17 07:08 PM 102      86.4 GB D         AVE01.test.local Local    app              Cloud
2019-10-16 07:07 PM 101      86.4 GB D         AVE01.test.local Local    app              Cloud
2019-10-15 07:09 PM 100      86.4 GB D         AVE01.test.local Local    app              Cloud
--------------- snip ---------------
2019-08-25 07:07 PM 53       86.4 GB DW        AVE01.test.local Local    app              Cloud
2019-08-24 07:07 PM 52       86.4 GB D         AVE01.test.local Local    app              Cloud
2019-08-23 07:07 PM 51       86.4 GB D         AVE01.test.local Local    app              Cloud
 
 

5. After being configured and run, it is now possible to verify task completion:

sysadmin@dd3300# data-movement show history 
       Start Time            End Time   Bytes Moved   Bytes Moved   Files Moved      Elapsed Time   State
                                         (Pre-comp)   (Post-comp)                      (hh:mm:ss)
-----------------   -----------------   -----------   -----------   -----------   ---------------   ---------
Nov 20 2019 20:00   Nov 20 2019 20:03      6.22 GiB     22.01 MiB            17   0 days 00:03:02   Completed
Nov 19 2019 20:00   Nov 19 2019 20:11      6.20 GiB      1.29 GiB            17   0 days 00:11:36   Completed
Nov 18 2019 17:21   Nov 19 2019 03:52      4.81 TiB     29.59 GiB          1026   0 days 10:31:01   Completed
Sep 24 2019 16:55   Sep 24 2019 16:55             0             0             0   0 days 00:00:00   Completed
Sep 24 2019 16:53   Sep 24 2019 16:53             0             0             0   0 days 00:00:00   Completed
-----------------   -----------------   -----------   -----------   -----------   ---------------   ---------
sysadmin@dd3300# data-movement status
Data-movement to cloud tier:
----------------------------
Data-movement was started on Nov 20 2019 20:00 and completed on Nov 20 2019 20:03
Moved (post-comp): 22.01 MiB, (pre-comp): 6.22 GiB,
Files inspected: 14227, Files eligible: 17, Files moved: 17, Files failed: 0

Data-movement recall:
---------------------
No recall operations found.
Use 'path all' option to list all recall jobs (up to the maximum supported limit).
sysadmin@dd3300# data-movement status to-tier cloud detailed
Data-movement to cloud tier:
----------------------------
Data-movement was started on Nov 20 2019 20:00 and completed on Nov 20 2019 20:03
Moved (post-comp): 22.01 MiB, (pre-comp): 6.22 GiB,
Files inspected: 14227, Files eligible: 17, Files moved: 17, Files failed: 0

Data-movement status for MTrees:
--------------------------------
MTree                              Files      Files   Files    Files   Bytes Moved   Bytes Moved   Destination
                               Inspected   Eligible   Moved   Failed    (Pre-comp)   (Post-comp)   Cloud Unit
----------------------------   ---------   --------   -----   ------   -----------   -----------   -----------
/data/col1/avamar-1562250622       14227         17      17        0      6.23 GiB     22.01 MiB   Avamar
----------------------------   ---------   --------   -----   ------   -----------   -----------   -----------
(*) Data-movement is in progress for the marked MTrees.

Files currently being moved:
----------------------------
Path Name   File Size       Logical   Destination   Elapsed Time
                        Bytes Moved   Cloud Unit        hh:mm:ss
---------   ---------   -----------   -----------   ------------
---------   ---------   -----------   -----------   ------------ 

Affected Products

Avamar

Products

Avamar, Avamar Server, Data Domain
Article Properties
Article Number: 000174845
Article Type: Solution
Last Modified: 22 Oct 2024
Version:  5
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.