Isilon: Falha no upgrade de firmware do nó no CMC
Summary: Os upgrades de firmware do nó para componentes do CMC podem encontrar uma falha/erro e precisar ser reinicializado a frio, ou a energia do nó drenada, para resolver esse problema.
This article applies to
This article does not apply to
This article is not tied to any specific product.
Not all product versions are identified in this article.
Symptoms
Durante um upgrade de firmware de nó, a atualização de firmware do CMC falha/apresenta erros.
No /var/log/isi_firmware.log do nó que apresentou falha, um resultado semelhante ao seguinte será exibido:
2017-05-11 15:18:58,991 ERROR [17093] : Node 6: CMCFirmware._update: update failed: error=65280 output: TARGET UPGRADE CAPABILITIES ------------------------------- HPM.1 version............0 Component 0 presence....[y] Component 1 presence....[y] Component 2 presence....[y] Component 3 presence....[y] Component 4 presence....[n] Component 5 presence....[n] Component 6 presence....[n] Component 7 presence....[y] Upgrade undesirable.....[n] Aut rollback override...[n] IPMC degraded...........[y] Defered activation......[n] Service affected........[n] Manual rollback.........[n] Automatic rollback......[n] Self test...............[n] Upgrade timeout.........[300 sec] Self test timeout.......[0 sec] Rollback timeout........[300 sec] Inaccessibility timeout.[300 sec] GENERAL PROPERTIES ------------------------------- Payload cold reset req....[n] Def. activation supported.[n] Comparison supported......[n] Preparation supported.....[y] Rollback supported........[y] Description string: CMC FW Current Version: Major: 00 Minor: 0b Aux : 00 00 00 00 Validating firmware image integrity...OK Performing preparation stage...TARGET UPGRADE CAPABILITIES ------------------------------- HPM.1 version............0 Component 0 presence....[y] Component 1 presence....[y] Component 2 presence....[y] Component 3 presence....[y] Component 4 presence....[n] Component 5 presence....[n] Component 6 presence....[n] Component 7 presence....[y] Upgrade undesirable.....[n] Aut rollback override...[n] IPMC degraded...........[y] Defered activation......[n] Service affected........[n] Manual rollback.........[n] Automatic rollback......[n] Self test...............[n] Upgrade timeout.........[300 sec] Self test timeout.......[0 sec] Rollback timeout........[300 sec] Inaccessibility timeout.[300 sec] GENERAL PROPERTIES ------------------------------- Payload cold reset req....[n] Def. activation supported.[n] Comparison supported......[n] Preparation supported.....[y] Rollback supported........[y] OK Error initiating upgrade action, compcode = ff Firmware upgrade procedure failed Error aborting upgrade, compcode = 80 ... ----------------------------------------------------------------- |ID | Name | Active | Upload Progress | Upload| Image | | | | Version |0% 50% 100%| Time | Size | |---|-----------|---------||----+----+----+----||-------|-------| ...Failed... -----------------------------------------------------------------
In OneFS 8.0, this would also cause output similar to the following for upgrade commands:
Isilon-1# isi upgrade nodes firmware progress list Lnns Device Old Version New Version Status ----------------------------------------------------------------------------------- ... 6 CMCSDR_Yeti 00.06 00.06 - 6 CMC_Yeti 00.0b 02.05 error 6 IsilonFPV1 UI.01.36 UI.01.36 - ... ----------------------------------------------------------------------------------- Total: 75 Isilon-1# isi upgrade nodes view 6 Node LNN: 6 Node Upgrade State: error Error Details: Failed Step: InstallFirmware Error Log: /ifs/.ifsvar/upgrade/Agent-6-000e1e591a20232c8a58011809d7e56ff3fb.log Last Upgrade Action: - Last Action Result: - Node Upgrade Progress: None Node OS Version: 8.0.0.4_build(53)style(5) Isilon-1# isi_upgrade_logs Copied log to /ifs/upgrade_error_logs/6_isi_Upgrade_Agent.py-InstallFirmware.log Isilon-1# cat /ifs/upgrade_error_logs/6_isi_Upgrade_Agent.py-InstallFirmware.log Starting /usr/sbin/isi_Upgrade_Agent.py InstallFirmware Verifying boot device mirrors are synced... 2017-05-11 15:18:42,876 DEBUG Execute Command: InstallFirmware 2017-05-11 15:18:59,005 ERROR Failed to update CMC_Yeti firmware on Node 6
Cause
Esse problema é causado porque o componente CMC não responde em tempo hábil durante a atualização do firmware do nó.
Resolution
O problema pode ser resolvido realizando uma reinicialização a frio dos componentes BMC e CMC no nó ou, se a reinicialização a frio não resolver o problema, é necessário desligar e drenar a energia do nó de acordo com https://support.emc.com/kb/466373 Se nenhuma das etapas resolver o problema, encaminhe para as equipes de suporte técnico do Isilon para análises adicionais.
Para redefinir o CMC, execute o seguinte comando no nó afetado:
/usr/bin/isi_hwtools/isi_ipmicmc -c -a cmc
Exemplo de resultado:
Para redefinir o CMC, execute o seguinte comando no nó afetado:
/usr/bin/isi_hwtools/isi_ipmicmc -c -a cmc
Exemplo de resultado:
Isilon-6# /usr/bin/isi_hwtools/isi_ipmicmc -c -a cmc Cold reset CMC Note - 15 sec wait for CMC... Checking if Firmware Version MSB is set MSB set in Firmware Version Waiting 5 seconds before trying again Success: MSB cleared in Firmware Version Checking self test results Success: self test passed
Once reset, retry the firmware update.
In OneFS versions prior to 8.0:
isi firmware update
In OneFS versions 8.0 and later:
Review the 'Node Upgrade State' for each node, identify the node that is in an error state, this is the node that the upgrade will be retried on:
isi upgrade nodes list
To retry the upgrade on the node in an error state, run the following command, replace LNN with the node that is showing in an error state:
isi upgrade retry-last-action LNN
Affected Products
IsilonProducts
Isilon, Isilon HD400, Isilon NL410, Isilon S210, Isilon X210, Isilon X410Article Properties
Article Number: 000050382
Article Type: Solution
Last Modified: 11 Oct 2024
Version: 3
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.