We're trying to create an offsite copy from an Isilon in a Remote Location to another Isilon at a Data Center. The remote location has very little bandwidth and it will probably take months to complete the baseline copy.
Netapp has SMTape option to seed the remote site offline. I'm relatively new to Isilon and was wondering if SyncIQ has any such feature.
VNX has a similar option where you can do tape-out when creating a replicator session. In the industry, it's sometimes referred to as "tape silvering".
That said, I don't believe that SyncIQ has such an option (unless it's been added recently). Why? Because low-bandwidth, high-latency links are far less common today between datacenters than they used to be. That said there are other ways to get the data there, and you can tell SyncIQ (in the CLI only) that the target directory exists already. Be aware though that it'll have to do a treewalk for the first full.
Thanks for the response Chris. That is exactly what we are looking for. Will you be able to point me to some sort of documentation or guidelines on how to accomplish that?
maybe an alternative way is to move the isilon (if not needed elseway) from remote to the Location of the current isilon, sync it, move the Isilon back again.
also if the bandwidth is that bad, you should calculate the replication-traffic and if you can ever Keep the copys in sync over that link.
If not: don't sync the isilons. search for alternatives.
i am running one of this "target compare initial sync" job right now on a directory with 12 millions files. It's been running for 2 weeks and only 50% done. I am sure a lot depends on the backend (NL-SAS, X410) in my case but something to consider if you have very tight RPO.
Indeed it's not for the faint-of-heart. With a high-file-count and small average file size it can certainly be faster to just copy it all over the WAN instead. The first pass through has to do a full treewalk and comparison on every file, so it's going to be pretty slow.
And as sluetze said above often times in a case like this DellEMC will deploy the DR cluster next to the production cluster to seed it, then pack it and ship it to the other side, and re-establish replication.
A truck can move data sometimes far faster than any other method.
Thanks all for your responses. We decided to replicate over Internet VPN using NAT. Based on the speeds, it looks like we will be able to complete the baseline in 2 weeks
So that's at line rate. Now maybe factoring in the latency let's say it's only 1/2 that good. That's just 17 days for the first full, then your first incremental or two will have a bunch of data to copy (depending on your change rate), but the changes are block level based upon the difference between 2 snapshots, and there is no tree walk required. So all in all not too bad. After the first incremental or two then you settle into a pretty steady rhythm. One thing to keep in mind is that you should think carefully about how frequently you run your SyncIQ jobs. For instance if you're running a sync ever 2 hours and it takes let's say 90 minutes to run, is there room to make it more frequent? Most people would say no at first glance but stop and think about it for a minute longer. If in those 90 minutes let's say it copied 10GB just for a round number. Assuming a constant rate of change if you ran ever 1 hour instead of 2, then you'd copy only 5GB every hour, and that job in theory would only take 1/2 of the time or 45 minutes. Want to run more frequently? How about copying 2.5GB every 30 minutes, with a job that completes in 22.5, and so-on. There is of course some overhead to each job, but not a ton. So keep that in mind is all I'm saying.
crklosterman
450 Posts
1371
2
Posted April 27th, 2017 06:00
VNX has a similar option where you can do tape-out when creating a replicator session. In the industry, it's sometimes referred to as "tape silvering".
That said, I don't believe that SyncIQ has such an option (unless it's been added recently). Why? Because low-bandwidth, high-latency links are far less common today between datacenters than they used to be. That said there are other ways to get the data there, and you can tell SyncIQ (in the CLI only) that the target directory exists already. Be aware though that it'll have to do a treewalk for the first full.
~Chris Klosterman
Principal SE, Datadobi
chris.klosterman@datadobi.com