IIQ 6.x migration stalled or not progressing

Cluster migration states that migration is in progress while nothing is occurring in the background. 

Multiple scenarios can lead to this such as rebooting during migration, loss of connection between cluster/VM, and so on...

Steps to Recover from Migration Interruption After System Reboot.

If an InsightIQ system  was rebooted during the migration and the process was interrupted, follow the steps below to resume migration and complete the upgrade successfully.

1. Restarting a Failed Migration 

If the migration process fails, one way to recover is by re-running the upgrade script. However, before doing so, a few preparatory steps are required to ensure that the script runs cleanly and the migration is re-triggered properly.  
 

Step 1: Verify the Resume Point

• Navigate to the following file:

/usr/share/storagemonitoring/logs/upgrade/tmp/resume_point.txt

• Open the file and look for the line containing:
  cluster_data_migration
• If this line is present, delete it to clear the previous migration checkpoint. 

Step 2: Remove the Upgrade Lock

• Delete the upgrade lock file to allow the upgrade script to proceed:
       rm /usr/share/storagemonitoring/logs/upgrade/tmp/upgrade.lock

Step 3: Re-run the Upgrade Script

• After completing the above steps, re-run the upgrade script.

This re-triggers the migration and upgrade process. There is no need to run any additional script for migration.

Step 4: Monitor migration logs, and check process is running

• ps auwx | grep cluster_data_migration.sh
• less /usr/share/storagemonitoring/logs/upgrade/insightiq_db_migration_restore_XXXXXX.log
• tail -f /usr/share/storagemonitoring/logs/clustermanagement/insightiq_cluster_migration.log

If this is an offline migration such as 4.x to 6.x, you will check the following location for logs: 

/usr/share/storagemonitoring/logs/offline_migration/insightiq_offline_migration.log