chappel02
1 Copper

Data Domain compression / de-dupe expectations

We just got a new DD160 and are starting to integrate it into our backup infrastructure. We have a very typical setup - about 40 virtualized windows servers, mostly file servers, SQL servers, exchange, print server, domain controllers, etc. We use Storage Craft Shadow Copy for our front end backup client. I configured the backup jobs to disable compression on the client side and pointed them to the DD160. We are over 50% utilization on capacity and still copying our data over, but only seeing about 2.3x compression / de-dupe. Is this normal? We were led to expect at least 10x space savings. I would have thought multiple backups of the same server OS would give better de-dupe results than this, and the data / user files would compress much more than 2x. What are we missing?

Thanks,

ch

0 Kudos
14 Replies
ble1
6 Indium

Re: Data Domain compression / de-dupe expectations

Which compression do you use on DD? Do you backup fulls or incrementals or...?

0 Kudos
chappel02
1 Copper

Re: Data Domain compression / de-dupe expectations

I'm doing full, uncompressed backups (I didn't copy over any archived backups, so I had nothing to do a differential/incremental with), and am using the default 'lz' compression on the DD160.

0 Kudos
ble1
6 Indium

Re: Data Domain compression / de-dupe expectations

One thing which might influence de-dupe ratio is of course encoding of the stream, but I doubt this is the case.  I can only assume that data copied so far was unique for some reason and if you say data didn't change, then stream format is such that is not very de-dupe friendly.  I happen to have one DD160 and backup is pretty much the same as you describe, but I get obviously better numbers:

sysadmin@dd160# filesys show compression

                     

From: 2014-04-04 16:00 To: 2014-04-11 16:00

                     

                  Pre-Comp   Post-Comp   Global-Comp   Local-Comp      Total-Comp

                     (GiB)       (GiB)        Factor       Factor          Factor

                                                                    (Reduction %)

---------------   --------   ---------   -----------   ----------   -------------

Currently Used:    14241.1      1208.0             -            -    11.8x (91.5)

Written:*                                                                       

  Last 7 days       4972.3       380.4          9.2x         1.4x    13.1x (92.3)

  Last 24 hrs        683.7        50.7          9.2x         1.5x    13.5x (92.6)

---------------   --------   ---------   -----------   ----------   -------------

* Does not include the effects of pre-comp file deletes/truncates

   since the last cleaning on 2014/04/06 05:13:07.

Key:                                                         

       Pre-Comp = Data written before compression            

       Post-Comp = Storage used after compression            

       Global-Comp Factor = Pre-Comp / (Size after de-dupe)  

       Local-Comp Factor = (Size after de-dupe) / Post-Comp  

       Total-Comp Factor = Pre-Comp / Post-Comp              

       Reduction % = ((Pre-Comp - Post-Comp) / Pre-Comp) * 100

0 Kudos
ble1
6 Indium

Re: Data Domain compression / de-dupe expectations

... and my filesys options are:

sysadmin@dd160# filesys option show

Option                            Value  

-------------------------------   --------

Local compression type            gzfast 

Marker-type                       auto   

Report-replica-as-writable        disabled

Current global compression type   9      

Staging reserve                   disabled

-------------------------------   --------

0 Kudos
PatrickBetts
3 Argentium

Re: Data Domain compression / de-dupe expectations

Hi chappel02,

I am a Data Domain TSE.  Could you email the serial number of the affected system so that I can look at the latest AutoSupports?  My email address is patrick.betts@emc.com.  I'll take a look and see if I can figure out what's going on.

Best Regards,

Patrick

0 Kudos
chappel02
1 Copper

Re: Re: Data Domain compression / de-dupe expectations

As I said, these are our initial copies of data, so there are no multiples of full backups to de-dupe, but I would think typical end-user data, databases and email would compress and de-dupe better than 2x, as would multiple installs of the same OS files.

I assume your DD160 is showing the results from ongoing weekly full backups?

for comparison:

sysadmin@DDBackup01# filesys sh compression

                    

From: 2014-04-04 12:00 To: 2014-04-11 12:00

                    

                  Pre-Comp  Post-Comp  Global-Comp  Local-Comp      Total-Comp

                    (GiB)      (GiB)        Factor      Factor          Factor

                                                                    (Reduction %)

---------------  --------  ---------  -----------  ----------  -------------

Currently Used:    5336.3      2484.7            -            -    2.1x (53.4)

Written:*                                                                      

  Last 7 days      7773.0      4002.4          1.6x        1.2x    1.9x (48.5)

  Last 24 hrs      4804.7      2150.2          1.8x        1.2x    2.2x (55.2)

---------------  --------  ---------  -----------  ----------  -------------

* Does not include the effects of pre-comp file deletes/truncates

  since the last cleaning on 2014/04/10 08:35:53.

Key:                                                        

      Pre-Comp = Data written before compression          

      Post-Comp = Storage used after compression          

      Global-Comp Factor = Pre-Comp / (Size after de-dupe)

      Local-Comp Factor = (Size after de-dupe) / Post-Comp

      Total-Comp Factor = Pre-Comp / Post-Comp            

      Reduction % = ((Pre-Comp - Post-Comp) / Pre-Comp) * 100

0 Kudos
ble1
6 Indium

Re: Data Domain compression / de-dupe expectations

Yes it is ongoing, but I would expect more from DBs and OS (which has many things static and repeated among different boxes).

0 Kudos
Highlighted
chappel02
1 Copper

Re: Re: Re: Data Domain compression / de-dupe expectations

Here are my options:

sysadmin@DDBackup01# filesys option show

Option                            Value  

-------------------------------   --------

Local compression type            lz     

Marker-type                       auto   

app-optimized-compression         none   

Report-replica-as-writable        disabled

Current global compression type   9      

Staging reserve                   disabled

-------------------------------   --------

0 Kudos
ble1
6 Indium

Re: Re: Re: Data Domain compression / de-dupe expectations

Perhaps Patrick can dig something up after you send him autosupport.

0 Kudos