dynamox
6 Thallium

SyncIQ - clean up after failback

Hello guys/gals,

Below is copy/paste from OneFS WebAdmin guide. I have gone thorugh multiple failover/failback on my virtual cluster without issue, steps are very simple. One thing that i don't undertand is why after we complete the failback process we do not get rid of the <replication-policy-name>_mirror  policy. It did its job, it pushed new/changed data back to primary cluster so why do we need it. Is it an over-site in the documention or there is a good reason to keep this policy around ?

Thank you

**********************************************

Procedure

  1. On the primary cluster, click Data Protection > SyncIQ > Policies .
  2. In the SyncIQ Policies table, in the row for a replication policy, from the Actions column, select Resync-prep.SyncIQ creates a mirror policy for each replication policy on the secondary cluster.SyncIQ names mirror policies according to the following pattern:<replication-policy-name>_mirror
  3. On the secondary cluster, replicate data to the primary cluster by using the mirror policies.You can replicate data either by manually starting the mirror policies or by modifying the mirror policies and specifying a schedule.
  4. Prevent clients from accessing the secondary cluster and then run each mirror policy again.To minimize impact to clients, it is recommended that you wait until client access is low before preventing client access to the cluster.
  5. On the primary cluster, click Data Protection > SyncIQ > Local Targets .
  6. In the SyncIQ Local Targets table, from the Actions column, select Allow Writes for each mirror policy.
  7. On the secondary cluster, click Data Protection > SyncIQ > Policies .
  8. In the SyncIQ Policies table, from the Actions column, select Resync-prep for each mirror policy.

After you finish

Redirect clients to begin accessing the primary cluster.

**********************************************

Tags (2)
0 Kudos
35 Replies
bhalilov1
2 Iron

Re: SyncIQ - clean up after failback

When the *_mirror policy is created on DR cluster, protection domain is created on your PROD cluster, you can see it if you run

isi_classic domain list

When you're finished and failed back to PROD, the domain is in Writable state. If you delete the *_mirror policy the domain also gets deleted. The next time you failover and run "isi sync recovery resync-prep" to failback, you will have to wait for that domain mark job to run again - it may take long time.

On the other side of the coin, if you decide to keep the mirror policy and the protection domain on your PROD cluster, you will not be able to mv files to the directory that's under the domain, even if it's in "Writable" state

0 Kudos
dynamox
6 Thallium

Re: SyncIQ - clean up after failback

Hi Burhan,

i guess i don't understand what happens in step 8 ? Why do we need to run resync-prep  ? My data has been copied from DR back to primary cluster via the mirror policy and now i am just ready to restart my  primary --> DR replication.  Does step 8 do something to DR cluster location to prepare it to become replication target again (makes it read/only) ?

Thank you

0 Kudos
Highlighted
bhalilov1
2 Iron

Re: SyncIQ - clean up after failback

Yes,

Step 6 changes your protection domain in PROD cluster from "Write Disabled" to Writable

if you run the forward policy at this point it will fail, since your DR side is still also Writable

in step 8 by running the resync-prep from PROD you change the state of the domain in DR to "Write Disabled"

0 Kudos
sluetze
3 Silver

Re: SyncIQ - clean up after failback

the mirror policy also leaves a snapshot (at least under 7.0.2.x) that grows like "nether world" (this americans with their censorship of common not harmful words like hell) - in our case several TB in a few weeks and is not needed, since it is the snapshot that gets created on the source of a replication. It is deleted automatically after one year. We started to delete it right after we failed back, to not waste unnecessary clusterspace

dynamox
6 Thallium

Re: SyncIQ - clean up after failback

Burhan Halilov wrote:

Yes,

Step 6 changes your protection domain in PROD cluster from "Write Disabled" to Writable

if you run the forward policy at this point it will fail, since your DR side is still also Writable

in step 8 by running the resync-prep from PROD you change the state of the domain in DR to "Write Disabled"

Burhan, this make sense but is there any reason to keep the mirror policy around after completing step 8 ?

0 Kudos
dynamox
6 Thallium

Re: SyncIQ - clean up after failback

sluetze wrote:

the mirror policy also leaves a snapshot (at least under 7.0.2.x) that grows like "nether world" (this americans with their censorship of common not harmful words like ****) - in our case several TB in a few weeks and is not needed, since it is the snapshot that gets created on the source of a replication. It is deleted automatically after one year. We started to delete it right after we failed back, to not waste unnecessary clusterspace

Do you remember the name of the policy, i want to check on my 7.1.x cluster.

0 Kudos
sluetze
3 Silver

Re: SyncIQ - clean up after failback

"SIQ-%ID-latest"

The %ID is the id of the policy.

i had an SR# with EMC and they said they are useless. Currently we are trying to either use our processtool for DR to automatically delete these snaps or we will create a feature request to let them delete automatically inside OneFS.

Regards

--sluetze

0 Kudos
johnsonka
2 Iron

Re: SyncIQ - clean up after failback

Hello Dynamox,

I conferred with the subject matter expert for SyncIQ in Support and he stated that the reason we do not recommend removing the mirror policy is it destroys the LIN map being used by it. So, if you were to have to redo the failover process it has to recreate the LIN map for the mirror policy from scratch. This can be a HUGE time penalty when your failover window is tight and there are millions of files to scan in to creating the new LIN map to be used by the new mirror policy once failover is complete.

I hope this answers some of your question, but please let me know if there is any other information that I can get for you! I am more than happy to help.

0 Kudos
dynamox
6 Thallium

Re: SyncIQ - clean up after failback

Katie,

I am still not following you, maybe you can work through the below steps with me and explain what's happening with these SIQ snapshots and then how the mirror policy is playing its role.

here is a test scenario:

policy name = upgrade_ir

1) I am replicating from primary cluster to secondary cluster. When i check on primary cluster i see snapshot SIQ-3d510f0edc31e18bdb2100dc1441306e-latest  and on secondary cluster i see snapshot SIQ-Failover-upgrade_ir-2015-08-11_19-18-35

This is what i consider my normal state.

************

2) i am failing over from primary cluster to secondary cluster. Upon completing that step on primary cluster i still see the same snapshot but on secondary cluster i now see two snapshots:

SIQ-3d510f0edc31e18bdb2100dc1441306e-restore-latest

SIQ-Failover-upgrade_ir-2015-08-11_19-33-34       

***********

3) I am failing back from secondary back to primary.  I go to primary cluster, select my policy and hit "Resync Prep". At this point i look on primary cluster and see these snapshots:

SIQ-3d510f0edc31e18bdb2100dc1441306e-latest         

SIQ-w2isilonpoc-upgrade_ir_mirror-2015-08-11_19-37-17

SIQ-w2isilonpoc-upgrade_ir_mirror-latest            

secondary cluster has on only this snapshot

SIQ-0050568f48352c87ca553e2004ae341d-latest

Now i am going to the secondary cluster and run upgrade_ir_mirror policy. When i look on primary cluster i see these snapshots:

SIQ-3d510f0edc31e18bdb2100dc1441306e-latest      

SIQ-Failover-upgrade_ir_mirror-2015-08-11_19-41-39

and on secondary

SIQ-0050568f48352c87ca553e2004ae341d-latest.

Next on primary cluster i go to Local Targets and select Write Enable for the upgrade_ir_mirror policy.  At this point on primary i see these snapshots

SIQ-3d510f0edc31e18bdb2100dc1441306e-latest       

SIQ-0050568f48352c87ca553e2004ae341d-restore-latest

SIQ-Failover-upgrade_ir_mirror-2015-08-11_19-48-25

and on secondary

SIQ-0050568f48352c87ca553e2004ae341d-latest

Finally i am going back to Secondary cluster and select Resync-prep for upgrade_ir_mirror policy. Primary cluster has this snapshot

SIQ-3d510f0edc31e18bdb2100dc1441306e-latest

and secondary

SIQ-0050568f48352c87ca553e2004ae341d-latest  

SIQ-n2isilonpoc-upgrade_ir-2015-08-11_19-51-57

SIQ-n2isilonpoc-upgrade_ir-latest            

So the question is why mirror policy needs to stay and why do i end up with 3 SIQ snapshots on secondary cluster compared to my "normal" state in 1)

Thank you very much for your time

0 Kudos