Highlighted
joejoejoejoe1
1 Nickel

How do YOU back up your Isilion?

Jump to solution

Hello,

I was curious to hear the various approaches being used in the wild to backup the Isilon system. I currently have the following equipment at my disposal but have found it disturbing/interesting/odd...etc..etc that there seems to be no way to put data sitting on an Isilon to my Data Domain system via NDMP.

OneFS Version: 7.1.0.0

Avamar Version: 7.0.1-61

DDOS Version: 5.4.0.7-401172

Now I see that there is a plug-in available within Avamar specifically for Isilon which is great but only if your Avamar system is sized in such a way to hold more then just Metadata/Catalogs..etc for backups that ultimately are just destined to go to a DD system at the end of the line.

I have come up with several "creative" ways that I could use to backup data sitting on an Isilon but none of them are what I would call elegant in any way shape or form.

Please share your experience and what your approach is for backing up this piece of equipment

Thanks for your time.

Labels (1)
Tags (1)
0 Kudos
1 Solution

Accepted Solutions
crklosterman
3 Argentium

Re: How do YOU back up your Isilion?

Jump to solution

When talking about NDMP and datadomain there are some trade-offs.  First an NDMP stream doesn't de-dupe as well as an NFS or SMB backup stream.  So there's that.  Second, when backing up a dataset with NDMP, in most cases you cannot restore it via NDMP to another vendor's system.  I see this pretty often with customers who have moved from NetApp to Isilon, and have backup tapes with NDMP streams from NetApp, but cannot restore that data on Isilon. 

Besides that there certainly are customers using EMC Networker with DD Boost, and writing to Data Domain the NDMP Streams.

Mounting via NFS and backing up that way is an option, but to be frank, only one I would consider as a last resort.  The reason is pretty simple.  Permissions.  When backing up over NFS, you're only going to get POSIX permissions, which most likely don't reflect the true composition of the security structure.  So when you perform a restore, yes you can do redirected restores, but you've lost some of the metadata associated with that file. And yes, most customers when they get to a certain cluster size, say 500TB-1PB stop backing up their clusters, or at least they stop backing up the entire cluster.  Instead they backup critical data sets, like EDI applications, or other business critical apps, and SyncIQ everything else.

The other thing that can help here is a full once, incremental forever strategy. 

Hope this helps.

~Chris Klosterman

chris.klosterman@emc.com

twitter: @croaking

Senior Solution Architect

EMC Isilon Offer & Enabelment Team

13 Replies
ed_wilts
2 Iron

Re: How do YOU back up your Isilion?

Jump to solution

We use SyncIQ to replicate the data to our offsite data center.

You could use a client to mount up the /ifs via NFS and back that up just like any other client.  I'm not a big fan of NDMP.

crklosterman
3 Argentium

Re: How do YOU back up your Isilion?

Jump to solution

When talking about NDMP and datadomain there are some trade-offs.  First an NDMP stream doesn't de-dupe as well as an NFS or SMB backup stream.  So there's that.  Second, when backing up a dataset with NDMP, in most cases you cannot restore it via NDMP to another vendor's system.  I see this pretty often with customers who have moved from NetApp to Isilon, and have backup tapes with NDMP streams from NetApp, but cannot restore that data on Isilon. 

Besides that there certainly are customers using EMC Networker with DD Boost, and writing to Data Domain the NDMP Streams.

Mounting via NFS and backing up that way is an option, but to be frank, only one I would consider as a last resort.  The reason is pretty simple.  Permissions.  When backing up over NFS, you're only going to get POSIX permissions, which most likely don't reflect the true composition of the security structure.  So when you perform a restore, yes you can do redirected restores, but you've lost some of the metadata associated with that file. And yes, most customers when they get to a certain cluster size, say 500TB-1PB stop backing up their clusters, or at least they stop backing up the entire cluster.  Instead they backup critical data sets, like EDI applications, or other business critical apps, and SyncIQ everything else.

The other thing that can help here is a full once, incremental forever strategy. 

Hope this helps.

~Chris Klosterman

chris.klosterman@emc.com

twitter: @croaking

Senior Solution Architect

EMC Isilon Offer & Enabelment Team

ed_wilts
2 Iron

Re: How do YOU back up your Isilion?

Jump to solution

The reason I suggested NFS was simply because that's we use our Isilons for - almost 100% NFS v3 - so an NFS-based backup strategy would catch it.  If you're strictly an SMB shop, backing up via NFS would be pretty foolish as Chris pointed out.  If you're 100% SMB, then backing via SMB shares might be an option.

I still prefer SyncIQ for our environment simply because it's not practical to walk petabyte file systems.    That's also how I found a few SyncIQ bugs

0 Kudos
sluetze
3 Silver

Re: How do YOU back up your Isilion?

Jump to solution


There as a presentation by Dr. Stefan Radtke, CTO EMEA about Backup Concepts. It has some nice comparison between backup possibilities.

It is available at the following link:

http://germany.emc.com/collateral/customer-profiles/isilon-backup-concepts.pdf

some deeper informations are available here:

http://stefanradtke.blogspot.de/2014/09/challenges-and-options-for-filesystem.html

Peter_Sero
4 Beryllium

Re: How do YOU back up your Isilion?

Jump to solution

The OneFS NDMP implementation does a pretty good job crawling large hierarchies,

due to extensive prefetching along the snapshots it processes. NFS/SMB based backup

clients would need some manual integration to use snapshots, and still couldn't

do that level of prefetching. Moreover, OneFS 7.1 NDMP can do snapshot-based diffs,

which are really fast.

If you can increase the number of concurrent NDMP jobs as your cluster is growing,

it can go quite far. For disaster recovery, we do nightly diffs of ca. 50 shares

containing 150 Mio files, half of which have metadata on SSD (3 X + 5 NL nodes),

with up to eight parallel NDMP jobs. And that's with 7.0.2 old-style crawling diffs!


So far we see no reason why linear increase of data, node count *and* NDMP jobs

should not keep working work for us. Partly because the number of our shares

(or backup-configurable subfolders) will increase so that we can run more jobs,

partly because tape (LTO) technology more or less keeps up with the growth of

individual shares/folders. YMMV of course.


Happy backing-upXXXXXXXXXrestoring!


-- Peter

0 Kudos
Adam_M1
1 Nickel

Re: How do YOU back up your Isilion?

Jump to solution

Interesting Read, I need to read up on the change-list feature

ps, the pdf has "EMC CONFIDENTIAL—INTERNAL USE ONLY" on every page, I hope it's ok to share It was a good read.

0 Kudos
umichklewis
4 Beryllium

Re: How do YOU back up your Isilion?

Jump to solution

For some of us, NDMP really is a good choice.  We configured an accelerator node, attached via FC to an LTO tape library.  We rely on 30 days snapshots for day-to-day user file recovery, then run weekly and monthly fulls to tape.  This has been an extremely cost-effective solution for us, and still meets our operational goals.

The only thing that would increase our solution would be to purchase a partner cluster, configure it to keep 60 days of snapshots and replicate to it via SyncIQ.  At that point, I would probably move my tape library to a third location, and backup the replication target to tape.  That way, I have instant recovery from local snapshots, longer-term recovery from the DR cluster, then archival recovery from geographically-distant tapes.

Full disclosure:  I'm a fan of the NDMP protocol, regardless of it's inability to restore across vendor solutions.  I manage several NAS products, some from multiple vendors, and NDMP lets me share resources (tape) where I need it. We've been able to provide three-way NDMP backups and restores for other organizations on Campus, who didn't have much experience with NDMP.

Peter_Sero
4 Beryllium

Re: How do YOU back up your Isilion?

Jump to solution

Karl, are you already using snapshot-based diffs for NDMP?

I have tried them out and found, they are fast but still we can't use them

for most of our production share because the full (= NDMP level 0) snapshots

will get too large.

You wrote that you retain regular snapshots for 30 days,

that should also cover the full backup cycles, right?

Cheers

-- Peter

0 Kudos
Jim_Hegner
2 Iron

Re: How do YOU back up your Isilion?

Jump to solution

Hi Everyone...

Our case is rather unusual ( I think).

We have a setup where we are implementing a 50 Node  X400 cluster with a total of 3 PB capacity.

And a Secondary cluster for replica site with NL-400's

Change rate of data 300 TB every day ( it is a HPC setup)

Daily, weekly and monthly backups

retention 90 days ( if possible)

The customer has a reqiurement for 4 hard copies of the data...so we are thinking NDMP to tape with networker for the 2 copies not on Isilon

Anyone care to give an idea on how to solve this small headache...

Cheers Jim

0 Kudos