lluiscle
2 Bronze

Networker NDMP Backup of NAS with millions of files

Hi ,

We have a Siebel that uses a VNX NAS share to store all it's files, due to implementation restrictions it uses only root folder to save all files and we don't have the right to change this.

Number of files is so big that ls even ls -f or ls -U wouldn't work, last time we counted with a script over 20.000.000 files.

As you can imagine with this scenario NDMP backups don't work in their optimal performance.

I think it takes 2/3 hours to list content before it starts to write (maybe I'm wrong it's only a feeling)

I wanted to know if any of you has a similar scenario and found a workaround for this .

If we had subfolder it would be easier as we could create different save sets.

I've considered creating a policy to save based on file creation date but as far as I know NDMP doesn't allow any flags to filter by date.

Data Domain, Networker Server and VNX are all on the same vlan but backups are starting to fail due to an increase on backup save time.

We're using NW 8.2.3.5 right now , NW server is windows .

Regards,

0 Kudos
3 Replies
ble1
6 Indium

Re: Networker NDMP Backup of NAS with millions of files

Ah, joy of SiebelFS.  I'm not sure now how many files we have on our end (I believe it is far less), but I remember that I had rather good performance with both FS backup (when on Linux; on HPUX it was bad) and via NDMP (however, using another filer vendor).  What you can try is export fs to backup server (or storage node is you use remote storage node), mount it there and try backup that way and measure if you get any difference compared to NDMP time.  On top of that, with fs approach you can also create multiple ssids to pick up multiple stream from same mnt point which should bring some parallel processing and have it done more quickly (can't be done with NDMP... plus due to update to local index table this will take some time more there).

0 Kudos
lluiscle
2 Bronze

Re: Networker NDMP Backup of NAS with millions of files

Hi , Thanks for the prompt answer, we were evaluating that option but after your reply I've moved forward, hopefully I'll have a Linux server to measure the difference between FS and NDMP.

Regarding parallel processing the issue is that affects production performance.

0 Kudos
ble1
6 Indium

Re: Networker NDMP Backup of NAS with millions of files

What if you create snapshot of the volume and mount snapshot for the backup?  In essence, NDMP backup does create first snapshot.  Here, you would create it manually and export it to storage node and do backup from there as NFS or CIFS mount (depending of OS, I guess NFS since you mentioned test with Linux).  This approach might have sense only if you find speeds you get to be better than with NDMP.

0 Kudos