Backup is too slow with large save set with more than 10 million files

Hi,

you can try to use the bigadm directive to check whether really the file system is the bottle neck:

https://emc--c.na5.visual.force.com/apex/KB_HowTo?id=kA0700000004KjG

Regards

Michael

First of all it is important that you use B2D - tape drives are no good as they permanently shoe-shine with these data rates.

Then, you do not have much choices:

  - either you compress (zip) the files at the client

  - or you run an image-style backup (block based backup or RAW partition backup)

This is pure NetWorker.

It will speed up if you think about a combination of NW and Avamar but this of course will need extra investments.

I would suggest you to make use of Block based backup which is designed to solve problems similar to what you are currently facing.

however this solution required the target device to be a disk type (AFTD or DataDomain), you can propose to add disk storage to your backup infra to create a AFTD and do a POC for Block based backup for your organization to approve the additional charges.

I also do agree of leveraging BBB. There could many causes also.

Bigasm can help knowing the bottleneck.

You can also divide into many client policies and run backup for individual savesets, and not all at once.

Hi Bingo,

please keep in mind that "compress (zip) the files at the client" makes only sense if neither ddboost nor deduplication is used (e. g. in case of having a DataDomain backup device).

Regards

Michael

@vakil

Actually, bigasm is more testing network and backup device speed than anything else - so I think the results are not valid in this case.

@kleinenbroich

I am not sure about ddboost - we use it on similar filesystem with no client compression and the throughput is rather slow as well. But keep in mind that it all depends on a lot of factors, so you must run true tests with your true data and your backup scenario anyway.to find the best method.