Manual expiration of savesets via gui?

Hi

there is a point to make here , deleting the savesets from NW side , does not mean that the disk space will be released momentarily ! yes this is a fact , the savesets deletion means that the savesets will be removed from Networker media DB , But the actual removal from disk only takes place as part of the space recovery procedure , this Space Recovery in only launched regularly by the master process NSRD (once per day) ; However , the Volume status / Flags  must not be "scan needed" or "Manual recycle"

to check the Volumes status run : mminfo -mvV , it will list All volumes status ad other info. as well

you can filter by volume type like : mminfo -mvV -q "type=Data Domain"

- if you are sure that there no blocking state for that volume , then run the : nsrstage -C -V , while tailing the daemon log , it should say something about the space recovery from that volume , even if it does not recovery anything it will say no space recovered from device xxxx / volume yyyy

Shareef

Bingo, I was actually ok with that. Normally, when I need to expire some data, I need its clone to go away as well.

Hi Shareef. I ran mminfo -mvV, and returned the following.

I was thinking that the nsrim -X actually ran the cleanup of the data on the datadomain. I was working with EMC on an SR, and was told that after running my nsrmm bat file containing the ssid's, and then running nsrim -X, that I should see space reclaimed under the "available" size on the DataDomain. Not the "cleanable", but "available". Is that supposed to be accurate?

The only reason why nsrim would skip removed files would be if you have scan flag set on volume - did you check for that? If you have it, you will also see message in the logs about it.

Toddman214 wrote: I was thinking that the nsrim -X actually ran the cleanup of the data on the datadomain. I was working with EMC on an SR, and was told that after running my nsrmm bat file containing the ssid's, and then running nsrim -X, that I should see space reclaimed under the "available" size on the DataDomain. Not the "cleanable", but "available". Is that supposed to be accurate?

No, you should only see space under cleanable rise. That space, approximately, is cleaned after data domain cleaning runs which is normally once per week.  You can run "filesys clean show schedule" on DD to see when your schedule is.

And thats where i'm seeing the issue. No matter what I have tried so far, or no matter how many savesets I removed (thousands!), the cleanable value should be going up, but it wont even budge one decimal point. Ill research that to see if there is some sort of flag on the volume. But, judging from my screenshot above, I see a "d" by one of the volumes under the flag column, but nothing else.

Hi

To answer the 1st question :

Q: I was thinking that the nsrim -X actually ran the cleanup of the data on the datadomain. I was working with EMC on an SR, and was told that after running my nsrmm bat file containing the ssid's, and then running nsrim -X, that I should see space reclaimed under the "available" size on the DataDomain. Not the "cleanable", but "available". Is that supposed to be accurate?

A>>> No that is not accurate , the savesets removal from Disk will never - ever affect the Available space momentarily , it will only affect the Cleanable Space (adding more cleanable space)  , then once DD File system clean-up is done the Cleanable space will affect (proportionally because there is always the deduplication factor) the Available space (adds more available space once file system cleanup is done claiming the deleted files disk space)

Q: And thats where i'm seeing the issue. No matter what I have tried so far, or no matter how many savesets I removed (thousands!), the cleanable value should be going up, but it wont even budge one decimal point. Ill research that to see if there is some sort of flag on the volume. But, judging from my screenshot above, I see a "d" by one of the volumes under the flag column, but nothing else.

A>>>> "d" means that he volume is dirty , and is what we call a disk volume when the savesets have been deleted from the media DB , But not yet removed from disk ; so once the space recovery process get done with that volume , the "d" should be removed

>>> the 2nd point here is the When ? when should this space recovery process run ?
it should run every 23~24 hours, just after NSRIM automatic run ; it does not have a certain command to push the automatic run , But you can initiate a volume cleanup by running "nsrstage -C -V

removed saveset XXXXXXXX from device YYYYYYY

........

>>> it is advised to let things flow the automatic way , unless there is an urgent definite need to manual intervention

the alarming point would be to see lots of expired saevsets that are not removed by the space recovery process

HTH

Regards

Shareef

nsrim does run every 23 hours unless you manipulate nsrim.prv file inside /nsr/mm for example.  At that point ssids are removed.  "d" stands only for dirty which means that volume is performing write operation at the time of mminfo query.  dirty flag does not influence nsrim - scan setting does.  I'm not sure what you mean by "automatic run" as there is automatic run and you can trigger it yourself by either running it manually or by removing nsrim.prv.

You might see save sets not removed entirely from DD when there is heavy read operations on it (eg. check Recover space operation and concurrent read operations), but even so you do see some space cleared - opposed to what you claim none of it was cleared.  So, without any doubt, I'm quite sure picture we have is not complete and there is something else going on in environment described (most likely scan flag set).

Command to get the ssid list:

mminfo -avot -q "volume=name_volume" -r "ssid,savetime,nsavetime,volume,name,ssbrowse,ssretent"

If you want ssid and its clone id use the below command:

mminfo -avot -q " volume=name_volume " -r "ssid,cloneid,savetime,nsavetime,volume"

For individual SSID deletion you can use:

nsrmm -d -S SSID
nsrim -X or nsrstage -C -V

Run the following commands to recycle a single SSID from a Data Domain or adv_file volume:

nsrmm -e [todays_date] -w [yesterdays_date] -o recyclable -y -S [ssid]
nsrim "X -V [volume_name]
nsrstage -C -V [volume_name]

Above commands free up the space of deleted savesets for AFTD or file device. For Data Domain DDBoost device, freeing up space requires Data Domain Cleaning operation. By default, Data Domain only operates disk cleaning once a week on Tuesday.

You can run Cleaning operation on Data Domain web GUI under Data Management > File System > Start Cleaning.

For deletion of multiple SSID's you can follow the procedure below for Windows:

1. Put all the ssid into file called ssid.txt
2. Create bat file with this line:
   For /F %%a in (ssid.txt) do nsrmm -d -y -S %%a
3. Run the batch file to complete the saveset deletion.

To free up the space by deleting savesets from a Data Domain or adv_file device.(Deleting multiple savesets using batch/shell script commands)

1. Using MMINFO, display a list of savesets that you would like to expire, below are common Command Syntax examples:
   - To display all savesets for a specific Client and two separate Volumes:
         mminfo -avot -q client=[client_hostname],volume=[volume1_name],volume=[volume2_name]
   - To display all non-browsable Data Domain and adv_file savesets (quotes are needed when using spaces):
         mminfo -avot -q type=Data Domain,type=adv_file,recoverable
     - To display all savesets for a specific Group and Date range (quotes are needed when using special characters):
         mminfo -avot -q "group=[group_name],sscreate>=[start_date],sscreate<=[end_date]"

1. To delete the savesets you have listed, you need to first pipe the output to a text file using -r ssid -xcl > ssid.lst :
        mminfo -avot -q type=Data Domain,type=adv_file,recoverable -r ssid -xcl > ssid.lst
        ** this will create a file ssid.lst in the current directory.
2. Once you have the savesets piped to a text file, you can now run a batch/shell script command to recycle them:
       WINDOWS SYNTAX:        for /f %A in (ssid.lst) do @nsrmm -e [todays_date] -w [yesterdays_date] -o recyclable -y -S %A
       WINDOWS EXAMPLE:    for /f %A in (ssid.lst) do @nsrmm -e 01/18/13 -w 01/17/13 -o recyclable -y -S %A
       LINUX SYNTAX:                 for x in `cat ./ssid.lst`; do nsrmm -e [todays_date] -w [yesterdays_date] -o recyclable -y -S $x; echo $x; done
       LINUX EXAMPLE:              for x in `cat ./ssid.lst`; do nsrmm -e 01/18/13 -w 01/17/13 -o recyclable -y -S $x; echo $x; done
       ** This command must be ran in the same directory where the ssid.lst is located.
       After the savesets have been marked recyclable, NetWorker will automatically delete them over time (generally 24-48 hours).

To manually scan disk volumes for change and delete recyclable savesets, perform the following commands:
Create a list of Data Domain and adv_file volumes:

mminfo -q "type=data domain,type=adv_file" -r volume -xcl > volume.lst

- Tell NetWorker to scan the volumes for change:
      WINDOWS EXAMPLE:    for /f %A in (volume.lst) do @nsrim -X -V %A
      LINUX EXAMPLE               for x in `cat ./volume.lst`; do nsrim -X -V $x; echo $x; done

- Tell NetWorker to start the saveset deletion process:
      WINDOWS EXAMPLE:    for /f %A in (volume.lst) do @nsrstage -C -V %A
      LINUX EXAMPLE               for x in `cat ./volume.lst`; do nsrstage -C -V $x; echo $x; done

For NetApp, the space will not immediately become available, depending on a couple of factors:
    Snapshot policy, if snapshots are being taken of the CIFS/NFS file system where the adv_file device resides.
    The performance of the filer, because space is reclaimed in a low priority background operation.
 
For Data Domain DDBoost device, freeing up space requires the Data Domain Cleaning operation to run.
    By default, Data Domain only operates disk cleaning once a week on Tuesday.
    You can manually start the process from the Data Domain Enterprise Manager GUI. Go to: Data Management > File System > Start Cleaning. 
    It is not recommended that you manually clean a Data Domain very often but rather let its weekly schedule clean it to preserve optimal de-duplication.

Thanks all. Moin, I've been through that document several times, but its confusing me. I've used both this section "For deletion of multiple SSID's you can follow the procedure below for Windows:" and this section "To free up the space by deleting savesets from a Data Domain or adv_file device.(Deleting multiple savesets using batch/shell script commands)".

Then I talk with EMC, Im getting different responses to what Im seeing in this article.

I hate to take the lazy route, but can someone tell me what commands to use once I have my ssid.txt list created, so that once the commands are run, I'll see new "cleanable" space on the datadomain. I'll then just run a manual cleaning in the DataDomain to make that cleanable space available. I am running DPS on Windows, but Im having my primary DataDomain replaced this Wednesday, Nov 4, and right now it is sitting at 99.1%. It needs to be at or below 90% before we can start the work., so I have a TON of saevsest to remove!

99.1% means trouble and cleanup of DD will takes ages.  In such case, I would rather allocate extra space somewhere else and divert backups going to so other target than DD while trying to clean DD.

Once you have ssids in file, you can remove them via script with with nsrmm command as noted before.  Once this is done, nsrim will mark those removed and then you can run DD cleanup.  But, with DD at 99.1%, I wonder how certain processes work as at those stages DD itself runs in special mode.  That's why it is far better to stick to instructions given by support.

Hrvoje, I was able to clean some space through the netbackup application, clearing some of the images that were backed up by that product. But, i still need to remove more. I'm at 94.9% this morning. Which nsrmm script should I run? I have have two of them.

For /F %%a in (ssid.txt) do nsrmm -d -y -S %%a

or

for /f %A in (ssid.txt) do @nsrmm -o recyclable -y -S %A

Todd

the 1st one deletes the savesets from media DB

the 2nd one expires the savesets and defers the deletion for the space recovery

in my opinion both are similar in effect = NW will mark a list of savesets for deletion , either by fully removing them from its media DB or by setting their retention flag to expire ; however you will need to force Space Recovery after either of the above steps

Force space recovery : nsrstage -C -V

this will force the server to delete the savesets from disk (provided that they have been either deleted or expired)

I am assuming that you are using only File Devices (this includes Data Domain type devices, or as we call it DDBoost)

PLEASE : be sure that you are not deleting any saveset that may be needed in the future as there is absolutely no way to recover the data back once the above steps are done

HTH

Shareef

Oh, and once I know which nsrmm command I should run, do i then run nsrstage -C -V against the volumes that contain the ssid's, OR do I run nsrim -X?  Sorry for all of the questions. I still have a lot to learn about this product, and really need to clear some space quickly on the datadomain.

As shareef said, they are similar.  I would go for first one.  After that you can run nsrim -X.

Thanks shareef,

So here a step-by-step of what I did. Please let me know where I am wrong.

Starting point. Available = 7.19. Cleanable = 1.21

Generated a ssid.txt file containing 2,437 ssid's that I need to remove, from 4 different volumes.

Ran batch file from same location containing For /F %%a in (ssid.txt) do nsrmm -d -y -S %%a. It ran for about 4 hours. I could see it going through each batch file.

Once completed, I ran nsrstage -C -V against each volume that had ssid's that I need to remove.

Each ran for a couple minutes, and went back to prompt.

I waiting about 20 minutes, and checked my DataDomain stats again. Here is what they look like now. Available has continued to fall, as backups are running, but cleanable only increased by about 40GB or so. This was probably because I ran a test cleanup from another Backup application sending to DataDomain.

Am I doing something wrong, or do I need to open a ticket with DataDomain?