Isilon: During upgrade to 9.1.x or later upgrade stuck in committing

Summary: During upgrade to 9.1.x or later upgrade stuck in committing. During upgrade the error "isi_antivirus_post_commit.py Inform Failed to rename tmp_scans table as Scans table with error there is already another table or index with this name: Scans " appeared in isi_upgrade_logs. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

Upgrade looks like it is still in a commit:

Upgrade Status:
Current Upgrade Activity: Commit in progress
 Cluster Upgrade State: committing
 Upgrade Process State: Done
 Upgrade Start Time: 2021-04-24T08:35:50.184000
 Current OS Version: 9.1.0.0_build(3)style(5)
 Upgrade OS Version: N/A
 Percent Complete: 100%
Nodes Progress:
Total Cluster Nodes: 38
 Nodes On Older OS: 0
 Nodes Upgraded: 38
Nodes Transitioning/Down: 0
LNN Version Status
--------------------------
1-38 9.1.0.0 committing
last update: 2021-04-25T19:58:25


The following error is observed: "isi_antivirus_post_commit.py Inform Failed to rename tmp_scans table as Scans table with error there is already another table or index with this name: Scans " when running "isi_upgrade_logs" command or reviewing the upgrade logs 
 

Cause

There are two problems being seen in this situation: 

  1. isi_antivirus_post_commit is responsible for changing the format of the antivirus engine to include the cava enhancements, when this is done, if the scan table is empty the system does not know how to handle it.
  2. The upgrade engine may still display the process as committing, when it is paused. The true status can be seen in the upgrade logs using the following command {if there is no node 1, change the db name to an existing node}: 
sqlite3 /ifs/.ifsvar/upgrade/logs/UpgradeLog-1.db "select * from LogEntry where Entry like '%paused%'"


Example below: 

# sqlite3 /ifs/.ifsvar/upgrade/logs/UpgradeLog-1.db "select * from LogEntry where Entry like '%paused%'"                                       
39|2021-05-07T14:19:31.508795Z|0050569341df0e73646091118d1e9e5772e8|41463:100185|/usr/sbin/isi_upgrade_supervisor|Inform|/b/mnt/src/isilon/bin/isi_upgrade/isi_upgrade_supervisor/ProcessCheck.h:66||Process is paused

 

Resolution

To work around this issue,

Check to see if the upgrade is paused and run the following command to resume upgrade activity: 
isi upgrade cluster resume
Otherwise run retry-last-action command: 
isi upgrade cluster retry-last-action --nodes=<all or affected nodes if in error>

Then check the status of the upgrade activity again.
Article Properties
Article Number: 000186093
Article Type: Solution
Last Modified: 22 ذو القعدة 1447
Version:  5
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.