bedwards2
1 Copper

Tracking Data Domain usage with Avamar and DPA?

Jump to solution

This is one of those weird ones. Is it a DPA question? Avamar? Data Domain? Really, it's sort of all of them. However, as I feel DPA should be able to help answer this sort of question, I'm going to start here and move it as needed.

The Objective:

Recent datacentre moves have shifted some Avamar workloads. All Avamar workloads write to one of our DD's. (nothing on the grid other than metadata).  The directive has come down from on high that we need to move between 1 and 3   data domain ES30 shelves from one location to another to help this production move. In itself, this is not  a problem. We will have a loaner DD that we will do the appropriate replication to/from in order to remove the shelves and rebuild the DD.

The Question:

EMC asked us to provide how much data is being taken up on the Data Domain by certain clients, groups, domains, anything other than just the overall DD utilization number. Why? This would allow us to determine what we can move off the existing DD, and determine how many shelves we can ultimately move, not to mention properly size the loaner DD coming in. Even if we could determine how much data is currrent, how much is in RETIRED, and how much has been replicated to this DD, that would help. 

The Difficulty:

There doesn't seem to be a good way to do this. EMC SE's have sort of punted the question back to us. "Tell us how much is there, specifically as you can, and we'll help size appropriately."

DPA reports, and avamar reports (ie: DPN summary) do not seem to be able to directly let me know how much data is on the DD in specific to anything granular. I can get avamar data sizes and deduplication levels by client from DPA.... but that doesn't translate well into knowing what's on the DD, and I can't even get that sort of data on RETIRED or REPLICATED sources.  I've got great reports on total filesystem capacity and trending on the DD thanks to DPA, but that solves nothing in specific.

This is something I've struggled with before. Networker is better able to answer these sorts of questions than Avamar, depending on how Networker/Boost is set up. However, data domains in general, and avamar in specific with DD, really doesn't seem to answer this question well. I figure something must know. If I delete a client's data in Avamar, the DD knows how much to clean during the next cleaning cycle... but what?

The Assumptions:

Avamar 7.0.2

Data Domain OS 5.4, single DDBoost storage unit from avamar. No other data sources for the DD. 

DPA 6.1SP2

So, how would you approach this? (apparently my response of "Raucous laughter and heavy drinking" isn't acceptable). How do you know what in specific is taking up x amount of space on your DD?


1 Solution

Accepted Solutions
DavidRussell1
3 Argentium

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

Hi Brendan, In DPA 6.2 a new tool was added called the Data Domain Processor Tool (separate download available via support.emc.com). This tool is installed on a separate server/vm and has 3 scripts to run. The first connects to a DD and outputs the info about all the files on it to a file. The 2nd script then takes this file as an input, parses through it, connects back to the DPA server and maps each file to the backup client that generated it, then groups the file sizes together per client and sends the info back into the DPA server. The 3rd script gathers additional data about the size and count of the files on the DD grouped by age ranges and also sends it back into the DPA server. This then allows you to run some reports in DPA over your DD which tells you how much space (pre-and post logical sizes) is being used by each client. The restrictions here is that DPA has to know about the job that generated the file on the DD (otherwise it can't map the file). HTH David

15 Replies
DavidRussell1
3 Argentium

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

Hi Brendan, In DPA 6.2 a new tool was added called the Data Domain Processor Tool (separate download available via support.emc.com). This tool is installed on a separate server/vm and has 3 scripts to run. The first connects to a DD and outputs the info about all the files on it to a file. The 2nd script then takes this file as an input, parses through it, connects back to the DPA server and maps each file to the backup client that generated it, then groups the file sizes together per client and sends the info back into the DPA server. The 3rd script gathers additional data about the size and count of the files on the DD grouped by age ranges and also sends it back into the DPA server. This then allows you to run some reports in DPA over your DD which tells you how much space (pre-and post logical sizes) is being used by each client. The restrictions here is that DPA has to know about the job that generated the file on the DD (otherwise it can't map the file). HTH David

DavidRussell1
3 Argentium

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

1.PNG.png

DavidRussell1
3 Argentium

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

2.PNG.png

DavidRussell1
3 Argentium

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

3.PNG.png

Lindsay2
1 Copper

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

Brendan

Get yourself a copy of DPA 6.2 SP1. It has a new tool call the Data Domain Processing Tool - that allows you determine which of the many client types is chewing up your DD space.

Raucous laughter should stop.

Heavy Drinking can start

bedwards2
1 Copper

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

This looks like interesting stuff! I'll check into it as soon as I'm able to.

0 Kudos
bedwards2
1 Copper

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

Of course, with my usual timing, currently the 6.2 SP1 binaries are unavailable from EMC until further notice, to boot...

0 Kudos
Bdizzle1
2 Bronze

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

Where is this tool located? I just upgraded to SP1 and cannot find it.

0 Kudos
Lindsay2
1 Copper

Re: Tracking Data Domain usage with Avamar and DPA?

Jump to solution

The tool can be found under the <install_directory>

In a folder called 'dataprocessor'

There are three components to the tool :

dd_scanner - which collates raw data

file_age - which processes data for file age

client_aggregation - which processes raw data per client

0 Kudos