How to free space on Data Domain for allready deleted save sets from networker database

Update:

I found this document:

How to manually delete save sets which are within retention period and no longer needed?

and here is the extract from this insturctions:

For individual SSID deletion you can use:

nsrmm -d -S SSID
nsrim -X or nsrstage -C -V

Run the following commands to recycle a single SSID from a Data Domain or adv_file volume:

nsrmm -e [todays_date] -w [yesterdays_date] -o recyclable -y -S [ssid]
nsrim -X -V [volume_name]
nsrstage -C -V [volume_name]

Above commands free up the space of deleted savesets for AFTD or file device. For Data Domain DDBoost device, freeing up space requires Data Domain Cleaning operation. By default, Data Domain only operates disk cleaning once a week on Tuesday. You can run Cleaning operation on Data Domain web GUI under Data Management > File System > Start Cleaning.

But this doesn't work. When I try to recycle certain SSID the command returns:

5875:nsrmm: save set "ssid" does not exist

Sorry but what you have written look very strange.

You want to delete a save set using

    nsrmm -d -S

This is o.k. As you mentioned, this deletes the SS from the MDB and - if it still exists - the CFI for that SS as well.

However

  nsrmm -e [todays_date] -w [yesterdays_date] -o recyclable -y -S [ssid]

looks weird as you identify the SS by the SSID. There is no need to specify neither browse or retention date.

You simple overrule the dates by specifying the status 'recyclable'. I don't even know if such combination of parameters is accepted at all. Setting a disk SS to recyclable will delete it (immediately or after a short delay) from the media.

You can enforce this automatism by starting the process immediately (nsrim). 'nsrstage -C' is not really ncessary.

But of course this is all NW - he does not really know the internal characteristics of a DD. That's why you have to initiate the 'garbage collection' on that device separately (or you just wait).

But this is really weird:

  >> When I try to recycle certain SSID the command returns: 5875:nsrmm: save set "ssid" does not exist <<

  - Please stay consistent with the terminology: You cannot recycle a single save set - only the whole media

  - What do you want to achieve: 're-adding' a save set where you already have deleted its MDB entries?

       How shall NW resolve the ssid now? - the MDB does not know it any longer!

This is when you could use 'scanner' - to scan the media/device for unknown save sets.

However, this only makes sense for tapes - as mentioned above, NW will delete the SSs from disks automatically.

So most likely, they are gone right after you issued the delete command.

marinko_dzale wrote: 5875:nsrmm: save set "ssid" does not exist

You get that when your input file has header which contains column title which is ssid.  Remove that first line and you should be fine.

Removing data from DD happens on next DD cleanup.  Check when that is with filesys clean show config command.  How much it is estimated to be cleaned you can see via df command.  And if next cleaning is too far away for you, you can run filesys clean start to kick it right away once nsrmm is done (I would suggest to run nsrim -X after that and once cleanup on devices is done, then proceed to DD part).

edit:

after I executed scanner -m "device_name" command

i took few SSID's and I marked them as recyclable with

nsrmm -e [todays_date] -w [yesterdays_date] -o recyclable -y -S [ssid]

and the executed this:

nsrim -X -V [volume_name]

nsrstage -C -V [volume_name]

this deleted arroung 160GB fom this volume

before I recycled thos few SSIDs I checked one by one to see if they are visible in networker console and there were no such SSIDs.

So if command scanner -m device_name

recreates entries in media database onlyI presume that scanner command only returns my deleted SSIDs vhich are still on volume but not visible in media and index db?

This is important to me beacuse I don't have list of deleted SSIDs.

is it possible pipie result of scanner command to a file ?

hi guys

thank you for you answers

I tryed to run thi command from my storage node server:

scanner -i device_name (recreates entries in client index and media database)

scanner -m device_name (recreates entries in media database only)

but the return was:

8907:scanner: media database not responding; retrying...

8908:scanner: media database not responding; aborting

free_detached_multibubfs(0x0000000002EE03A0) called

I also executed both commands on my networker server (not on storage node) and

for command with -m switch it was ok

C:\Windows\system32>scanner -m rd=dr1ms21s.sb.corp:dr1dd.sb.corp_File_01

8909:scanner: using 'rd=dr1ms21s.sb.corp:dr1dd.sb.corp_File_01' as the device name

8936:scanner: scanning Data Domain disk File.001 on rd=dr1ms21s.sb.corp:dr1dd.sb.corp_File_01

8939:scanner: Data Domain disk File.001 already exists in the media index

8786:scanner: ssid 880987836: 31 MB, 184 file(s)

29485:scanner: ssid 126679248: scan complete

8786:scanner: ssid 126679248: 1 KB, 0 file(s)

29485:scanner: ssid 2709333808: scan complete

8786:scanner: ssid 2709333808: 1 KB, 0 file(s)

29485:scanner: ssid 3296215742: scan complete

8786:scanner: ssid 3296215742: 1 KB, 0 file(s)

29485:scanner: ssid 2512093016: scan complete

8786:scanner: ssid 2512093016: 1 KB, 0 file(s)

29485:scanner: ssid 829248710: scan complete

8786:scanner: ssid 829248710: 1 KB, 0 file(s)

29485:scanner: ssid 1083956106: scan complete

8786:scanner: ssid 1083956106: 1 KB, 0 file(s)

29485:scanner: ssid 2191079538: scan complete

8786:scanner: ssid 2191079538: 1 KB, 0 file(s)

29485:scanner: ssid 2291112834: scan complete

8786:scanner: ssid 2291112834: 129 KB, 71 file(s)

8786:scanner: ssid 1166027720: 815 KB, 48 file(s)

8761:scanner: done with Data Domain disk File.001

free_detached_multibubfs(0x000000000281F900) called

but for command with -i switch the return was:

C:\Windows\system32>scanner -i rd=dr1ms21s.sb.corp:dr1dd.sb.corp_File_01

8909:scanner: using 'rd=dr1ms21s.sb.corp:dr1dd.sb.corp_File_01' as the device name

8936:scanner: scanning Data Domain disk File.001 on rd=dr1ms21s.sb.corp:dr1dd.sb.corp_File_01

8939:scanner: Data Domain disk File.001 already exists in the media index

29485:scanner: ssid 1657059774: scan complete

8786:scanner: ssid 1657059774: 1 KB, 0 file(s)

29485:scanner: ssid 1539360093: scan complete

8786:scanner: ssid 1539360093: 1 KB, 0 file(s)

29485:scanner: ssid 2260434806: scan complete

8786:scanner: ssid 2260434806: 1 KB, 0 file(s)

).

107512:scanner: Retrieving statistics for '/r16ms161s/File_01/00/38/7b533bef-00000006-e3c78ba6-57c78ba6-09205000-27668d5

6' failed [5028] ([ 3776] [1884] Tue Mar 21 14:24:50 2017

        ddp_stat() failed, Path //r16ms161s/File_01/00/38/7b533bef-00000006-e3c78ba6-57c78ba6-09205000-27668d56, Err: 50

28-unrecoverable rpc failure

).

107512:scanner: Retrieving statistics for '/r16ms161s/File_01/00/39/f08e0d45-00000006-9f9ead2b-579ead2b-03625000-27668d5

6' failed [5028] ([ 3776] [1884] Tue Mar 21 14:24:50 2017

        ddp_stat() failed, Path //r16ms161s/File_01/00/39/f08e0d45-00000006-9f9ead2b-579ead2b-03625000-27668d56, Err: 50

28-unrecoverable rpc failure

).

107512:scanner: Retrieving statistics for '/r16ms161s/File_01/00/12/6b63166a-00000006-45802bc8-58802bc8-17bc5000-27668d5

6' failed [5028] ([ 3776] [1884] Tue Mar 21 14:24:50 2017

        ddp_stat() failed, Path //r16ms161s/File_01/00/12/6b63166a-00000006-45802bc8-58802bc8-17bc5000-27668d56, Err: 50

28-unrecoverable rpc failure

).

8761:scanner: done with Data Domain disk File.001

free_detached_multibubfs(0x0000000002B2F900) called

Hello.

I think this case is going to a complex way... Ussual situation is like this:

1. Networker is backing up normally

2. Nobody pays attention to data domain

3. Networker stops backups

4. Now, admin can see that Data Domain is on 100%

5. Admin manually deletes old savesets and run nsrim, nsrstage

6. Data Domain shows cleanable space.

7. Admin run cleaning in data domain (and wait for 2 or 3 days more, while ends)

8. All is running again, and lesson is learned.

Some step in the middle, if environment has tape or something else... admin redirect backups there.

Now, my question is: why are you running scanner? Why are you trying to recreate media/fileindex databases?.

NOTE: if you run scanner on data domain, it goes to read only mode. Then, you have to unmount the volume, change to appendable, and remount it.When volume is read only, cannot clean space.

If your savesets are not present in media index, you need to focus your attention in data domain in order to find the reason why  cannot clean deleted savesets.

Why would you run scanner command if you wish to get rid of data?  (hint: on media server you need to run it with -s as otherwise it will assume it runs on server already)

@hrvoje crvelin

beacuse I think taht those savesets that I deleted fom index and media db are not deleted from DD beacuse cleaning was executed 6 times during 15 days

And if I'm right and I if I get them back them then I can recycle those ssids and then delete them again.

I'm missing lot's of knowledge for Networker and DD so can you please tel me this?

If I delete SSID with nsrmm -d -S "ssid/cloneid" I will delete it from database

and than If I do nsrstage -C -V "volume on which is this ssid I deleted" then this will free space on Data Domain or not?

or the space from DD will be cleand after DD cleaning task ?

Hello. Calm down.

If I delete SSID with nsrmm -d -S "ssid/cloneid" I will delete it from database? YES

If I do nsrstage -C -V "volume on which is this ssid I deleted" then this will free space on Data Domain or not?YES

But... take a look if your data domain device is on read only mode. If so... Data Domain doesn't clean space.

@marinko

You said:

"If I delete SSID with nsrmm -d -S "ssid/cloneid" I will delete it from database"

Yes, this is true. But you also need to consider the drive/media type:

  - on a tape, you cannot delete the SS (you cannot delete/'zero' a section on tape)

  - on a disk device/media, each save set is stored in a single file. And this can of course be deleted at any time.

      When exactly, depends on NW/on the NW operation for the specific device type (FTD/AFTD).

  - on a DD device/media you deal with 2 issues:

       - You have NW who interacts with the front end (with the admin

       - You have DDOS which ensures that unused space will be re-released in the background.

         This of course is beyond NW's accessibility. As a standard NW admin, you have no access any longer.

    These are two independent processes.

You also said:

"and than If I do nsrstage -C -V "volume on which is this ssid I deleted" then this will free space on Data Domain or not?"

I never used nsrstage on volumes just on save sets. But it does not really as a volume is nothing else but a bunch of save sets. In general, the above also applies here:

  - You will notice the migration immediately in NW (front end)

  - DDOS will re-organize the data in the background later (background)

This would even be the case if you recycle/relabel a DD media.

ledugarte wrote: If I do nsrstage -C -V "volume on which is this ssid I deleted" then this will free space on Data Domain or not?YES

Actually no if using DD.  Then segments from that ssid will be marked for deletion (unique ones), but then won't be removed until next DD cleanup (scheduled or ad hoc).

There is one condition which will skip nsrim run (and nsrclone/nsrstage) to remove ssids from logical device and that is when volume has scan flag (can be verified via mminfo -q scan -r volume -  you don't wish to see anything reported back).

There is a video example for Networker 9 at how to free the space with all the steps needed. At 5 minutes https://www.youtube.com/watch?v=VBJ-zG2oWS4 it talks through the steps / commands need to check the Datadomain. The principle would be the same for other versions / backup software's that backs up to a Datadomain

Step1:
On the Volume, left click to show the Save Sets and the SSIDs.

Step2:
Select the backups you want to DELETE. You can select everything or just the ones you want.
After selecting, press CTRL + C and paste it into an excel table.

Step3:
In excel, leave only the important column, which in this case is the column containing the SSID. Than add the command lines that will be used in the CLI (nsrmm -d -S "ssid" -y)

Step4:
Open a SSH connection to your NVA server and paste (at once) the list made in excel.
After run copied lines, run:
nsrim -X
exit

Step5:
Open a SSH connection to your DD server.
Use the command bellow to display the Cleanable space:
filesys show space

Finally run the command bellow:
filesys clean start
(This command can take a few hours to complete and can be accompanied with the filesys clean watch command.)Step1

step2step3step4step5