PowerScale: Smartsync's SYNCIQ Policy Migration Error for One-to-many or Cascade Replication

SmartSync's SyncIQ policy migration fails with one of two errors:

1. Error Message: Migration blocked: Source record snapshot (<snapshot-id>) does not match latest dataset snapshot (<snapshot-id>) for the source basepath (<source-base-path>). This source basepath may act as a target for an existing SmartSync policy. Hence, the SIQ policy migration can't proceed unless the two snapshots are identical. Start the SIQ policy to match the latest dataset snapshot (<snapshot-id>) in source.
2. Error Message: Migration blocked: Source record snapshot (<snapshot-id>) is older than the snapshot (<snapshot-id>) associated with the latest dataset (<dataset details>) for the source basepath (<source-base-path>). Replicate to the latest snapshot or above to proceed with the SIQ policy migration.

Error 1 occurs when the Source Basepath has pre-existing dataset(s) which are marked as a target dataset by SmartSync. And the associated snapshot being used for migration, used by the SyncIQ Policy's source record is not the same one used by the latest dataset for the specific source-basepath.

Error 2 occurs when the Source Basepath has pre-existing dataset(s) which are not marked as a target dataset by SmartSync. And the associated snapshot being used for migration, used by the SyncIQ Policy's source record is not greater than or equal to the snapshot which used by the latest dataset for the specific source-basepath.

If the error has already occurred, go to Recovery Steps section.

Preemptive Migration Steps

Precautionary steps to take before the errors have already shown up in the migration job. Before starting the migration job, perform the following steps to avoid the above‑mentioned errors:

Fan-out configuration (A → B + A → C)

1. Step 1: Identify your SyncIQ policy Documentum Interactive Delivery Services. For example:

   1. Policy A → B:
      # isi sync policies view <pol_a_b> | head -2
   2. Policy A → C:
      # isi sync policies view <pol_a_c> | head -2

2. Step 2: the source records directory on Cluster A:

   # cd /ifs/.ifsvar/modules/tsm/config/source_records/

3. Step 3: List and view the XML file corresponding to each policy ID, and note the latest-snap-id value:

   1. Check A→B snapshot (the resulting ID will be AX in next step):
      # grep '<latest-snap-id>' <pol_a_b_ID>.xml

   2. Check A→C snapshot (the resulting ID will be AY in next step):
      # grep '<latest-snap-id>' <pol_a_c_ID>.xml

4. Step 4: Compare the latest-snap-id values. Migrate the policy with the older (lower)latest-snap-id first:
   If the A to B snapshot (AX) is older than the A to C snapshot (AY), for example: AX = snapID 100,AY = snapID150.
   Then migrate the A to B policy first.
   
   If the A to C snapshot (AY) is older than the A to B snapshot (AX), for example: AX = snapID 150,AY = snapID100.
   Then migrate the A to C policy first.
   
   If both policies use the same snapshot, for example: AX = 100,AY = 100.
   Either policy can be migrated first. The second policy migration reuses the dataset.

Chained (cascaded) configuration (A → B → C)

1. Synchronization should be fully completed, ensuring all clusters involved in policy chaining reflect an identical data.

2. Disable all SyncIQ policies in the chain before proceeding with migration. This prevents any new incremental SyncIQ jobs from being triggered while the migration is in progress. For example, in an A→B→C chain, disable both the A→B and B→C policies.

3. Important: Migration must always begin from the root (source) cluster. For any intermediate cluster (such as B in a 3‑cluster setup), all policies whose source basepath is also a target for another SmartSync policy must use the same (identical) snapshot as the latest dataset for that basepath. If the snapshots do not match, the migration job will pause(not permanently blocked). Recovery steps added below in Recovery Steps section.

   Before migrating chained policies, ensure that the B→C SyncIQ policy uses snapshot‑based synchronization with the same snapshot pattern produced by the A→B policy on Cluster B. This ensures both policies share the same snapshot baseline on the relay cluster.

4. If the B→C SyncIQ policy was not originally set up with snapshot‑based synchronization, update it before migration using the following steps:

   1. Step 1: Verify the A→B policy creates archive target snapshots (Target Snapshot Archive: should show Yes) as well with a naming pattern other than the default naming pattern of SIQ‑*.

      # isi sync policies view pol_wJ9n-a2b | grep -E "Target Snapshot Archive|Target Snapshot Pattern" Target Snapshot Archive: No Target Snapshot Pattern: SIQ-%{SrcCluster}-%{PolicyName}-%Y-%m-%d_%H-%M-%S

      As you would notice in the above example, Target Snapshot Archive is set to"No"and the pattern is the default one with a prefix of SIQ‑*.

   2. Step 2: Modify the A→ B policy to create archive snapshot with a different pattern other than the default one:

      # isi sync policies modify <A_to_B_policy> --target-snapshot-archive True --target-snapshot-pattern 'snapB-%{PolicyName}-%Y-%m-%d'

# isi sync policies view pol_wJ9n-a2b | grep -E "Target Snapshot Archive|Target Snapshot Pattern" Target Snapshot Archive: Yes Target Snapshot Pattern: snapB-%{PolicyName}-%Y-%m-%d
      

   3. Step 3: Modify the B→C policy to use --schedule=when-snapshot-taken:

      # isi sync policies modify <B_to_C_policy> --schedule=when-snapshot-taken

   4. Step 4: Trigger the A→B policy's job to ensure that the B→C Policy catches up on it and reuses the archive snapshot created by A‑>B; you can verify by running the reports command.

      # isi sync job start <A_to_B_policy>

# isi sync reports list

      Wait for both A→B policy's job to complete and in turn, that will trigger B→C policy's job. Once both the jobs are complete, validate that the archive snapshot created by A→B policy's latest run is reused by the latest job of B→C using the following command:

      # cat /ifs/.ifsvar/modules/tsm/config/target_records/<A_to_B_policy_Id>.xml | grep latest-archive-snap 
   <latest-archive-snap-alias>621</latest-archive-snap-alias> 
         <latest-archive-snap>625</latest-archive-snap>

# cat /ifs/.ifsvar/modules/tsm/config/source_records/<B_to_C_policy_Id>.xml | grep latest-snap-id
           <latest-snap-id>625</latest-snap-id>
   <restore-latest-snap-id>0</restore-latest-snap-id>
      

      As you would notice above, the latest-snap-id for B→ C policy is same as the latest‑archive‑snap for A→ B. Which is the state for chained policy's migration to go through successfully.

Recovery Migration Steps

If migration has already been attempted and the migration job goes into a PAUSED state, follow these recovery steps:

Fan-out configuration

When a SyncIQ policy migration job fails with the error, the corresponding SyncIQ policy is enabled back. The mitigation steps are:

1. Step 1: Run the SyncIQ policy to catch up:
   # isi sync job start <policy-name>
   
2. Step 2: Wait for the new job run to finish successfully and show up in the reports output:
   # isi sync reports list --policy-name <policy-name>
3. Step 3: Resume the SmartSync Migration job which had paused:
   # isi dm jobs resume <job_id>

Chained (cascaded) configuration

When a SyncIQ policy migration job fails with the error, the corresponding SyncIQ policy is enabled back. The mitigation steps are:

1. Step 1: Modify the SyncIQ policy using command:
   # isi sync policies modify <B_to_C_policy> --schedule=when-snapshot-taken
2. Step 2: Run A‑>B Smart Sync policy using command (assuming the upstream policy is already migrated):
   # isi dm policies modify <policy_name> --run-now=yes
   It will automatically get the snapshot synchronized with the latest dataset snapshot on the source basepath.
3. Step 3: Resume the migration job using command:
   # isi dm jobs resume <job_id>