Compression, Encryption and Dedupe - What's the deal??

Question

Hello DPS world, While delivering a tailored Networker / DD training today, a class participant asked me, What is the deal with sending compressed, encrypted data to Data Domain? Thought I would share the thoughts and discussion. PowerProtect DD does variable block deduplication, meaning all files are segmented in to 4K-12K blocks and compared against existing blocks and only the unique ones are stored and duplicates are discarded. So maximum benefit can be achieved by sending the same data over and over to DD, which is what happens with backups mostly. Considering a 5-10% change rate, the rest is deduped on a daily / weekly basis.  However when we present compressed / encrypted data to the DD's Dedupe algorithm, these two have a tendency to change the bit sequences in the blocks there by making it look like new blocks each time to PowerProtect DD and hence DD is forced to store the unique blocks, even though technically they belong to the same data, just jumbled in a different way. The end result? Dedupe suffers and we end up using more capacity on the DD. So can we not compress or encrypt data backed up to DD? Of course we can. DD by default compresses all data that is backed up to it, and we could even choose between LZ, GZ and GZ fast algorithms to do that. PowerProtect DD also supports inline Data Encryption at rest and when enabled, the data is deduped first, then compressed, encrypted and written to disk, which is more effective and efficient both for performance and Capacity!! The student liked the explanation! Hope you guys do too! Please post your questions below! Guru

Dell-RobS · Answer

Great explanation Guru!

Does it matter what product is sending the data to the PowerProtect DD? For example, are there differences in what's supported by the PowerProtect DD when using NetWorker vs. Avamar for example, or is it strictly a capability of the PowerProtect DD?

Dell-GururajS · Answer

Great question Rob! Compression and Encryption are both features that are native to PowerProtect DD. So all data that is written to DD can be compressed and Encrypted irrespective of backup software in use. Additionally both Avamar and Networker also support data encryption over the wire, so the data in the process of being backed up is also secure!! Hope that answers your question.

Dell-RobS · Answer

Awesome, that definitely answers my question!  Thank you!

Data Protection Education

Was this post helpful?