Article Number: 333438

printer Print mail Email

Avamar Capacity Management: How to delete or expire backups in bulk with the "modify-snapups" tool

Primary Product: Avamar

Product: Avamar Server more...

Last Published: 27 May 2020

Article Type: Break Fix

Published Status: Online

Version: 12

Avamar Capacity Management: How to delete or expire backups in bulk with the "modify-snapups" tool

Article Content

Issue


The purpose of this article is to document how to use the modify-snapups tool on an Avamar system to help with some of the following challenges.
  • The Administrator wants to delete backups in bulk to resolve or prevent a high capacity situation.
  • The system is becoming full and, although retention policies are reduced, this action affects only new backups. It is also necessary to alter the expiration of existing backups.
  • Existing backups were set with incorrect expiration dates.
  • Business requirements change and the organisation needs to store more or fewer backups than originally planned for.
    Cause
    Resolution
    The 'modify-snapups.sh' utility is a script which is present by default, on the Avamar Utility Node under /usr/local/avamar/bin.

    The utility replaces the scripts 'expire-snapups' and 'delete-snapups', which were earlier forms of the tool. Both of these earlier scripts are deprecated and the commands now directly call modify-snapups, if run (see item #1 in Notes).

    Expire and Delete modes:    
    modify-snapups is run using either the expire mode or the delete mode.
     
    modify-snapups --help
    modify-snapups - modify snapups
    Version: 7.1.0-370 (1.11)
    Usage: modify-snapups [--mode={delete|expire}] [--help] [options ]
    This program does nothing when invoked as modify-snapups.
    Instead, invoke this program as delete-snapups or expire-snapups,
    or else with --mode={delete|expire} before any other options.

    The flags that are passed to each mode are almost identical and are viewed by appending --help to the command. (see item #2 in Notes)

    What does the script do?  
    Running modify-snapups on its own is safe as it does not make any changes to the Avamar server.

    When run with correct arguments, the utility generates a script containing a list of commands. Running the script runs the list of commands in serial, in order to change the expiration dates of each backup or to delete it.

    The commands are generated according to the flags passed to modify-snapups.  

    A user can decide whether to include backups:      
    • Within a certain date range (--after / --before)
    • Under a certain domain (--domain)
    • For a certain client
    • Whether to include internal Avamar database backups for AVI, EM, MC
    • Whether to include backups on the replication target

    Example commands

    Example 1: Check all the backups under 'mydomain' and create a script which expires those created before 1 March 2015.
     
    modify-snapups --mode=expire --domain=/mydomain --before=2015-03-01 > expire-backups-output-script.txt

    The script produces a list of mccli backup edit commands to MODIFY (which, in this case, is to expire) the appropriate backups. Those not matching the criteria are also listed but are prefixed with a hash (#) and the description CONSERVE:   
     
    # MODIFY    Snapup date: 2015-02-17 08:04:30 GMT Lab#: 418     Replicated: N/A
    echo running: "mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=418 --created=2015-02-17 --expiration=2015-05-18"
    mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=418 --created=2015-02-17 --expiration=2015-05-18
    # CONSERVE Snapup date: 2015-04-17 08:05:18 IST Lab#: 457     Replicated: N/A
    # echo running: "mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=457 --created=2015-04-17 --expiration=2015-07-16"
    # mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=457 --created=2015-04-17 --expiration=2015-07-16
    # CONSERVE Snapup date: 2015-04-16 08:03:46 IST Lab#: 456     Replicated: N/A
    # echo running: "mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=456 --created=2015-04-16 --expiration=2015-07-15"
    # mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=456 --created=2015-04-16 --expiration=2015-07-15

    Example 2: Delete all backups under mydomain which were created more than one month ago.
     
    modify-snapups --mode=delete --domain=/mydomain --before='1 month ago' > output-script.txt

    Use proper dates as in example #1 but, just to illustrate, here are used 'friendly' Linux date notations (1 month ago).

    The script contains output as shown below (truncated for space reasons). In the header, the script explains exactly what operation it is carrying out and the parameters that are in force. The script was run on 17th April 2015.
    # ======================================================================
    # CLIENT: /mydomain/server
    #     Operation: DELETE selected snapups
    #     After: "June 1 1999 00:00:00" (928191600) Tue Jun  1 00:00:00 IST 1999
    #     Before: "1 month ago" (1426614045) Tue Mar 17 17:40:45 GMT 2015
    # ======================================================================
    # *** NO SNAPUPS FOUND ***
    # ======================================================================
    # CLIENT: /mydomain/myclient.avamar.com
    #     Operation: DELETE selected snapups
    #     After: "June 1 1999 00:00:00" (928191600) Tue Jun  1 00:00:00 IST 1999
    #     Before: "1 month ago" (1426614045) Tue Mar 17 17:40:45 GMT 2015
    # ======================================================================
    # CONSERVE Snapup date: 2015-03-18 08:03:31 GMT Lab#: 437     Replicated: N/A
    # echo running: "mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=437 --created=2015-03-18 --force=true"
    # mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=437 --created=2015-03-18 --force=true
    # MODIFY    Snapup date: 2015-03-16 08:03:57 GMT Lab#: 435     Replicated: N/A
    echo running: "mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=435 --created=2015-03-16 --force=true"
    mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=435 --created=2015-03-16 --force=true
    # MODIFY    Snapup date: 2015-03-13 08:01:30 GMT Lab#: 434     Replicated: N/A
    echo running: "mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=434 --created=2015-03-13 --force=true"
    mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=434 --created=2015-03-13 --force=true
    # MODIFY    Snapup date: 2015-03-12 08:09:20 GMT Lab#: 433     Replicated: N/A
    echo running: "mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=433 --created=2015-03-12 --force=true"
    mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=433 --created=2015-03-12 --force=true

    Backups that are listed with MODIFY are altered with the 'mccli backup delete' command. These backups all have 'created' dates prior to 1 month ago (17 March 2015).

    Example 3: Extend the expiration of existing backups.
    The following command takes backups that are created between 1 and 15 May 2018 for clients under the /clients domain and creates a script which, when run, replaces the expiration of those backups so that they expire 100 days after the backup was created.
     
    modify-snapups --domain=/clients/ --mode=expire --after='2018-05-01' --before='2018-05-15'  --days=100 > expire-backups-output-script.txt

    .
    8<----------------------SNIP------------------8<
    .
    # ======================================================================
    # CLIENT: /clients/server.asl.lab.emc.com
    #     Operation: EXPIRE selected snapups
    #     After: "2018-05-01" (1525158000) Tue May  1 00:00:00 PDT 2018
    #     Before: "2018-05-15 " (1526367600) Tue May 15 00:00:00 PDT 2018
    # ======================================================================
    # MODIFY    Snapup date: 2018-05-07 22:00:52 PDT   Lab#: 498     Replicated: N/A
    echo running: "mccli backup edit --domain=/clients --name=server --labelNum=498 --created=2018-05-07 --expiration=2018-08-15"
    mccli backup edit --domain=/clients --name=server --labelNum=498 --created=2018-05-07 --expiration=2018-08-15
    # MODIFY    Snapup date: 2018-05-06 22:00:46 PDT   Lab#: 497     Replicated: N/A
    echo running: "mccli backup edit --domain=/clients --name=server --labelNum=497 --created=2018-05-06 --expiration=2018-08-14"
    mccli backup edit --domain=/clients --name=server --labelNum=497 --created=2018-05-06 --expiration=2018-08-14
    8<----------------------SNIP------------------8< .
    # CONSERVE  Snapup date: 2018-06-25 22:00:41 PDT   Lab#: 501     Replicated: N/A
    # echo running: "mccli backup edit --domain=/clients --name=server --labelNum=501 --created=2018-06-25 --expiration=2018-10-03"
    # mccli backup edit --domain=/clients --name=server --labelNum=501 --created=2018-06-25 --expiration=2018-10-03
    # CONSERVE  Snapup date: 2018-04-30 22:00:57 PDT   Lab#: 491     Replicated: N/A
    # echo running: "mccli backup edit --domain=/clients --name=server --labelNum=491 --created=2018-04-30 --expiration=2018-08-08"
    # mccli backup edit --domain=/clients --name=server --labelNum=491 --created=2018-04-30 --expiration=2018-08-08


    Running the script to modify / delete backups
    After creating the script, check the contents carefully. When happy with the actions which are to be carried out, set the script to be executable.

    For example, create a script to delete all backups under /mydomain which are older than 6 months.

    Set the script to be executable and check the permissions:    
     
    admin@utility:~/modsnap/>: chmod +x delete-backups-mydomain-before-6monthsago.txt
    admin@utility:~/modsnap/>:  ls -l
    -rwx------ 1 admin admin 15016 Apr 17 18:55 delete-backups-mydomain-before-6monthsago.txt

    Run the script. Redirect the output to another file to record the changes made.
     
    admin@utility:~/modsnap/>: ./delete-backups-mydomain-before-6monthsago.txt
    running: mccli backup delete --domain=/mydomain --name=myclient.customer.com --labelNum=323 --created=2014-09-25 --force=true
    0,22553,Backup deleted.
    Attribute  Value
    ---------- --------------------------------------
    path       /mydomain/myclient.customer.com
    retention  N
    labelnum   323
    createtime 2014-09-25 18:07:04 IST
    plugin     3001

    If the script ran to completion and is run a second time, see:    
    1,22552,Backup does not exist

    If the script takes a long time to run and must be halted, it can be run a second time. To avoid rechecking previously deleted backups, edit the script manually to delete the commands already processed. Alternatively, regenerate the script again.

    Generating and running the script can take a significant amount of time. It is helpful to 'divide and conquer' by splitting the work into smaller periods of time or individual domains.
    Notes

    Item #1

     
    ls -ls /usr/local/avamar/bin/ | grep modify-snapups
         0 lrwxrwxrwx 1 root  root         14 Nov 28 19:42 delete-snapups -> modify-snapups
         0 lrwxrwxrwx 1 root  root         14 Nov 28 19:42 expire-snapups -> modify-snapups
        24 -rwxr-xr-x 1 root  root      24185 Sep  8 23:46 modify-snapups

    The term 'snapup' is an obsolete Avamar terminology which can be taken to mean a backup. It is a hybrid of the words snapshot and backup.

    Item #2
    Full documentation on how to use delete and expire modes is available by appending --help to the command.
     
    modify-snapups --mode=expire --help
    modify-snapups --mode=delete --help

    Item #3
    Be aware of the following article. See this video which is published on YouTube:    
     

     

    Issue


    The purpose of this article is to document how to use the modify-snapups tool on an Avamar system to help with some of the following challenges.
    • The Administrator wants to delete backups in bulk to resolve or prevent a high capacity situation.
    • The system is becoming full and, although retention policies are reduced, this action affects only new backups. It is also necessary to alter the expiration of existing backups.
    • Existing backups were set with incorrect expiration dates.
    • Business requirements change and the organisation needs to store more or fewer backups than originally planned for.
      Cause
      Resolution

      The 'modify-snapups.sh' utility is a script which is present by default, on the Avamar Utility Node under /usr/local/avamar/bin.

      The utility replaces the scripts 'expire-snapups' and 'delete-snapups', which were earlier forms of the tool. Both of these earlier scripts are deprecated and the commands now directly call modify-snapups, if run (see item #1 in Notes).

      Expire and Delete modes:    
      modify-snapups is run using either the expire mode or the delete mode.
       
      modify-snapups --help
      modify-snapups - modify snapups
      Version: 7.1.0-370 (1.11)
      Usage: modify-snapups [--mode={delete|expire}] [--help] [options ]
      This program does nothing when invoked as modify-snapups.
      Instead, invoke this program as delete-snapups or expire-snapups,
      or else with --mode={delete|expire} before any other options.

      The flags that are passed to each mode are almost identical and are viewed by appending --help to the command. (see item #2 in Notes)

      What does the script do?  
      Running modify-snapups on its own is safe as it does not make any changes to the Avamar server.

      When run with correct arguments, the utility generates a script containing a list of commands. Running the script runs the list of commands in serial, in order to change the expiration dates of each backup or to delete it.

      The commands are generated according to the flags passed to modify-snapups.  

      A user can decide whether to include backups:      
      • Within a certain date range (--after / --before)
      • Under a certain domain (--domain)
      • For a certain client
      • Whether to include internal Avamar database backups for AVI, EM, MC
      • Whether to include backups on the replication target

      Example commands

      Example 1: Check all the backups under 'mydomain' and create a script which expires those created before 1 March 2015.
       
      modify-snapups --mode=expire --domain=/mydomain --before=2015-03-01 > expire-backups-output-script.txt

      The script produces a list of mccli backup edit commands to MODIFY (which, in this case, is to expire) the appropriate backups. Those not matching the criteria are also listed but are prefixed with a hash (#) and the description CONSERVE:   
       
      # MODIFY    Snapup date: 2015-02-17 08:04:30 GMT Lab#: 418     Replicated: N/A
      echo running: "mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=418 --created=2015-02-17 --expiration=2015-05-18"
      mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=418 --created=2015-02-17 --expiration=2015-05-18
      # CONSERVE Snapup date: 2015-04-17 08:05:18 IST Lab#: 457     Replicated: N/A
      # echo running: "mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=457 --created=2015-04-17 --expiration=2015-07-16"
      # mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=457 --created=2015-04-17 --expiration=2015-07-16
      # CONSERVE Snapup date: 2015-04-16 08:03:46 IST Lab#: 456     Replicated: N/A
      # echo running: "mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=456 --created=2015-04-16 --expiration=2015-07-15"
      # mccli backup edit --domain=/mydomain --name=myclient.avamar.com --labelNum=456 --created=2015-04-16 --expiration=2015-07-15

      Example 2: Delete all backups under mydomain which were created more than one month ago.
       
      modify-snapups --mode=delete --domain=/mydomain --before='1 month ago' > output-script.txt

      Use proper dates as in example #1 but, just to illustrate, here are used 'friendly' Linux date notations (1 month ago).

      The script contains output as shown below (truncated for space reasons). In the header, the script explains exactly what operation it is carrying out and the parameters that are in force. The script was run on 17th April 2015.
      # ======================================================================
      # CLIENT: /mydomain/server
      #     Operation: DELETE selected snapups
      #     After: "June 1 1999 00:00:00" (928191600) Tue Jun  1 00:00:00 IST 1999
      #     Before: "1 month ago" (1426614045) Tue Mar 17 17:40:45 GMT 2015
      # ======================================================================
      # *** NO SNAPUPS FOUND ***
      # ======================================================================
      # CLIENT: /mydomain/myclient.avamar.com
      #     Operation: DELETE selected snapups
      #     After: "June 1 1999 00:00:00" (928191600) Tue Jun  1 00:00:00 IST 1999
      #     Before: "1 month ago" (1426614045) Tue Mar 17 17:40:45 GMT 2015
      # ======================================================================
      # CONSERVE Snapup date: 2015-03-18 08:03:31 GMT Lab#: 437     Replicated: N/A
      # echo running: "mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=437 --created=2015-03-18 --force=true"
      # mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=437 --created=2015-03-18 --force=true
      # MODIFY    Snapup date: 2015-03-16 08:03:57 GMT Lab#: 435     Replicated: N/A
      echo running: "mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=435 --created=2015-03-16 --force=true"
      mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=435 --created=2015-03-16 --force=true
      # MODIFY    Snapup date: 2015-03-13 08:01:30 GMT Lab#: 434     Replicated: N/A
      echo running: "mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=434 --created=2015-03-13 --force=true"
      mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=434 --created=2015-03-13 --force=true
      # MODIFY    Snapup date: 2015-03-12 08:09:20 GMT Lab#: 433     Replicated: N/A
      echo running: "mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=433 --created=2015-03-12 --force=true"
      mccli backup delete --domain=/mydomain --name=myclient.avamar.com --labelNum=433 --created=2015-03-12 --force=true

      Backups that are listed with MODIFY are altered with the 'mccli backup delete' command. These backups all have 'created' dates prior to 1 month ago (17 March 2015).

      Example 3: Extend the expiration of existing backups.
      The following command takes backups that are created between 1 and 15 May 2018 for clients under the /clients domain and creates a script which, when run, replaces the expiration of those backups so that they expire 100 days after the backup was created.
       
      modify-snapups --domain=/clients/ --mode=expire --after='2018-05-01' --before='2018-05-15'  --days=100 > expire-backups-output-script.txt

      .
      8<----------------------SNIP------------------8<
      .
      # ======================================================================
      # CLIENT: /clients/server.asl.lab.emc.com
      #     Operation: EXPIRE selected snapups
      #     After: "2018-05-01" (1525158000) Tue May  1 00:00:00 PDT 2018
      #     Before: "2018-05-15 " (1526367600) Tue May 15 00:00:00 PDT 2018
      # ======================================================================
      # MODIFY    Snapup date: 2018-05-07 22:00:52 PDT   Lab#: 498     Replicated: N/A
      echo running: "mccli backup edit --domain=/clients --name=server --labelNum=498 --created=2018-05-07 --expiration=2018-08-15"
      mccli backup edit --domain=/clients --name=server --labelNum=498 --created=2018-05-07 --expiration=2018-08-15
      # MODIFY    Snapup date: 2018-05-06 22:00:46 PDT   Lab#: 497     Replicated: N/A
      echo running: "mccli backup edit --domain=/clients --name=server --labelNum=497 --created=2018-05-06 --expiration=2018-08-14"
      mccli backup edit --domain=/clients --name=server --labelNum=497 --created=2018-05-06 --expiration=2018-08-14
      8<----------------------SNIP------------------8< .
      # CONSERVE  Snapup date: 2018-06-25 22:00:41 PDT   Lab#: 501     Replicated: N/A
      # echo running: "mccli backup edit --domain=/clients --name=server --labelNum=501 --created=2018-06-25 --expiration=2018-10-03"
      # mccli backup edit --domain=/clients --name=server --labelNum=501 --created=2018-06-25 --expiration=2018-10-03
      # CONSERVE  Snapup date: 2018-04-30 22:00:57 PDT   Lab#: 491     Replicated: N/A
      # echo running: "mccli backup edit --domain=/clients --name=server --labelNum=491 --created=2018-04-30 --expiration=2018-08-08"
      # mccli backup edit --domain=/clients --name=server --labelNum=491 --created=2018-04-30 --expiration=2018-08-08


      Running the script to modify / delete backups
      After creating the script, check the contents carefully. When happy with the actions which are to be carried out, set the script to be executable.

      For example, create a script to delete all backups under /mydomain which are older than 6 months.

      Set the script to be executable and check the permissions:    
       
      admin@utility:~/modsnap/>: chmod +x delete-backups-mydomain-before-6monthsago.txt
      admin@utility:~/modsnap/>:  ls -l
      -rwx------ 1 admin admin 15016 Apr 17 18:55 delete-backups-mydomain-before-6monthsago.txt

      Run the script. Redirect the output to another file to record the changes made.
       
      admin@utility:~/modsnap/>: ./delete-backups-mydomain-before-6monthsago.txt
      running: mccli backup delete --domain=/mydomain --name=myclient.customer.com --labelNum=323 --created=2014-09-25 --force=true
      0,22553,Backup deleted.
      Attribute  Value
      ---------- --------------------------------------
      path       /mydomain/myclient.customer.com
      retention  N
      labelnum   323
      createtime 2014-09-25 18:07:04 IST
      plugin     3001

      If the script ran to completion and is run a second time, see:    
      1,22552,Backup does not exist

      If the script takes a long time to run and must be halted, it can be run a second time. To avoid rechecking previously deleted backups, edit the script manually to delete the commands already processed. Alternatively, regenerate the script again.

      Generating and running the script can take a significant amount of time. It is helpful to 'divide and conquer' by splitting the work into smaller periods of time or individual domains.

      Notes

      Item #1
       
      ls -ls /usr/local/avamar/bin/ | grep modify-snapups
           0 lrwxrwxrwx 1 root  root         14 Nov 28 19:42 delete-snapups -> modify-snapups
           0 lrwxrwxrwx 1 root  root         14 Nov 28 19:42 expire-snapups -> modify-snapups
          24 -rwxr-xr-x 1 root  root      24185 Sep  8 23:46 modify-snapups

      The term 'snapup' is an obsolete Avamar terminology which can be taken to mean a backup. It is a hybrid of the words snapshot and backup.

      Item #2
      Full documentation on how to use delete and expire modes is available by appending --help to the command.
       
      modify-snapups --mode=expire --help
      modify-snapups --mode=delete --help

      Item #3
      Be aware of the following article. See this video which is published on YouTube:    
       

       

      Article Attachments

      Attachments

      Attachments

      Article Properties

      First Published

      Fri Feb 05 2016 18:19:31 GMT

      First Published

      Fri Feb 05 2016 18:19:31 GMT

      Rate this article

      Accurate
      Useful
      Easy to understand
      Was this article helpful?
      0/3000 characters