UNSOLVED
P
pittperson
96 Posts
0
1355
June 29th, 2009 11:00
Does dedupe have any affects on backups?
The subject says it all. With dedupe turned on for a file system are there any negative impacts to an NDMP backup process? For example, while duplication is in process, will backups fail?
Hi,
I'm afraid the documentation leaves a lot to be desired in describing where the limitations are here.
I'm currently implementing the backup solution for a pair of Celerras which have DeDupe turned on for all file systems.
I can take VBB (NVB) backups at Full/Level-1/Level-8 without a hint of trouble.
When it comes to restoring data I can do a "full, destructive restore" from a "Level-Full" backup (as described in the "NDMP backups for Celerra" book), but it seems that neither a "destructive restore"( block stream), nor a "file-level restore" (Single file, or indexed restore) are possible from backups taken at any other backup level.
If this is really the case, then the NDMP implementation should reject backup requests at any level other than "Full" if the File-system had De-dupe turned on.
I've got a support case open to get the official line on this, but if any forum folk can share their experiences it'd be much appreciated.
From
Will
Rainer_EMC
6 Operator
•
8645 Posts
560
0
Posted November 23rd, 2010 10:00
Hi Will,
take a look at the release notes – I think the restrictions on VBB and deduplication are described there.
Basically you cannot restore deduplicated files from a VBB backup – other (non-deduplicated files) from that backup should restore fine.
If you would rather want the functionality of not allowing any file level restores from a VBB backup I suggest to use the Product Enhancement Request from on Powerlink to input that to product management.
Rainer
Hi Rainer,
Thanks for the post, but I think you have mis-interpreted my requirement.
I'm fully aware that "Single file restore" is not supported from VBB backups of a DeDupe file-system.
However, I'm interested in Disaster Recovery - pulling whole backup sets off tape to restore the system after a major incident (i.e. Data Center fire). I have read through the Dart release notes (as you suggested), the "NDMP backup on Cellerra" guide, and the "DeDupe on Celerra" guide, and NONE OF THEM mention this as a limitation. (all DART-6.0 documentation)
I think the lack of clarity in the documentation is around the "Full Destructive Restore". Nowhere is it made clear that this is the ONLY restore option for VBB/DeDuped file-systems, OR that it is ONLY possible to read-in a Level-FULL backup using this method.
I had assumed that The restore process for any VBB backups (at whatever level) would be the same - and the only one documented is the "Full Destructive Restore". However it seems that any NON-full backup must be restored as files (rather than data-blocks), which is not allowed for backups of DeDuped data.......
I've got a support case open on this as well, so I'll see if that brings any further insights into the issue.
From
Will
Hi,
We have tested the NDMP backup on one of our Test Deduplicated FileSystem.It is taking FULL Data backup.
i.e. If 20 GB Data Reciding in a Deduplicated format, but the actual size is 80 GB, then it will Backup 80GB.
We are using EMC Networker Backup Software.
Hi Ash,
Sounds like you're probably not using the VBB/NVB feature of the Celerra NDMP implementation. If you're not using this, then the backup tool (PAX?) will read the whole file-system (like tar would under UNIX), and send all the files it finds off to tape. In that case, what you're seeing is expected behavior.
To get the backups running using VBB in stead, set your "Backup Command" attribute of the NetWorker Client to "nsrndmp_save -T vbb".
From
Will
Rainer_EMC
6 Operator
•
8645 Posts
560
0
Posted November 24th, 2010 05:00
are you using NDMP PAX ? what DART version?
Rainer_EMC
6 Operator
•
8645 Posts
560
0
Posted November 24th, 2010 07:00
you should be getting the benefits of space-reduced NDMP backup so that files are sent in its compressed from to the tape
the overall size could be different than on the file system though since single-instancing doesnt carry over to the backup - for example if you had 10 times the same file that on the deduplicated file system only exists once it will be written to the tape ten times.
If you are sure it isnt this or a reporting error I would suggest to open a service case
Just note that the Celerra is the *only* NAS system that will carry over deduplication to the backup - all other vendor's systems will always reduplicate / uncompress before sending to tape.
Rainer
gbarretoxx1
2 Intern
•
366 Posts
560
0
Posted June 29th, 2009 21:00
please, consult the documentation "Using Celerra Data Deduplication" available on powerlink.
On NDMP backups the files need to be reduplicated, so the backup performance might be little impacted, but the user access is rarely affected.
Network (LAN) and Celerra NDMP (non-VBB) backups
When backed up over the network using CIFS or NFS, space-reduced files are reduplicated to their original size for transfer to the backup application, although the data is not reduplicated on disk. The benefits of the space-saving storage efficiency realized in the production file system will not flow through to backups when using network-based or PAX-based NDMP backups of Celerra deduplicated file systems. In addition, when restoring files from a PAX-based NDMP backup, the file will be restored as a normal file and will not be deduplicated. Therefore, the restored file will require more file system space after the restore than it was consuming when backed up. Depending on the amount of available space within the file system, restoring previously deduplicated files from tape could potentially consume all free space within the file system.
Note :
Accessing deduplicated files requires more resources than accessing normal files, and the read rate of these files may be less than that of normal files. Because the data being deduplicated is typically cold, user access will likely not be affected. However, PAX-based NDMP backups, which must reduplicate the files during the backup process, will be slowed when backing up deduplicated files. This will be particularly noticeable for small files.
Gustavo Barreto.