Start a Conversation

This post is more than 5 years old

Solved!

Go to Solution

2446

July 10th, 2009 03:00

Large amount of small files takes huge amount of time to backup

Hi Guys,

we have a FS in the TB that takes days to backup.

The FS is made up of millions of small files, I was wondering if there is a way of speeding it up What would you guys reccomend?

Thanks
2dam

49 Posts

July 12th, 2009 22:00

Only problem with SnapImage is if you wish to do a full disk, or full directory restore. You really need a VTL or Advanced File Type device with SnapImage.

244 Posts

July 10th, 2009 03:00

There is always a problem with backing up millions of files. You can try to backup raw devices (but then you have to recover the whole volume to recover single file). You can also try to divide backup source to send more than one stream:
let's say you have to backup "/large" (or E:\large), try to define more than one saveset ie.:
/large/dir1
/large/dir2 etc.
than you can start more than one stream of backup at the time.

85 Posts

July 10th, 2009 03:00

Version of NW we are using is 7.3.4 on Win 2003sp2

87 Posts

July 10th, 2009 07:00

You could also consider the NetWorker PowerSnap suite, which can leverage snapshot capabilities.

tl

1.1K Posts

July 10th, 2009 07:00

This is a common problem with no easy solution - it is a result of Networker generating more data to track the files than the files themselves. You could look at doing a raw backup of the disk but that can be painful to get back. You could look at splitting the files up so that you backup a different part each night which does not speed things up but spreads out the time of the backup.

45 Posts

July 10th, 2009 10:00

You can consider use Snapimage with NDMP Client. Work in a lot of customer with this problem. Can backup a block level and recover is a file level.

186 Posts

July 13th, 2009 00:00

Ok so i think i know what your hinting at here and if so i have seen maybe the same problem.

IS tyour file server connected to shared storage "SAN" on an ethernet network or a Fabric solution?

is your storage node the same server as the management server?

I have had a situation where backing up across the network from a client has had poor or slow backup speeds due to offload of data at the storage node.

You might want to check and see what the Network cad is doing on the storage node and if it is offloading to a fabric solution to stream to the tape library this could be your bottle neck.

We had this problem and i put in a 4 port nic and teamed it with load balancing to let it offload to the fabric to then stream to the tape drives.

Another question is are you backing up from one shared storage in one data centre to a tape library in a second data centre? if so i would be looking to shapshot the SAN and present the LUN directly to the storage node at the second DC is the infrastructure is correct you should not hit copper anywhere across the 2 dc's and have a full fabric connection to the tape drives.

Let me know.
No Events found!

Top