Isilon: CMC에서 노드 펌웨어 업그레이드 실패

Summary: CMC 구성 요소에 대한 노드 펌웨어 업그레이드에 장애/오류가 발생할 수 있으며 이 문제를 해결하려면 콜드 리셋하거나 노드 전원을 방전해야 합니다.

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms



노드 펌웨어 업그레이드 중에 CMC 펌웨어 업데이트 실패/오류

발생장애가 발생한 노드의 /var/log/isi_firmware.log에 다음과 유사한 출력이 표시됩니다. 
2017-05-11 15:18:58,991 ERROR    [17093] : Node 6: CMCFirmware._update: update failed: error=65280 output:
TARGET UPGRADE CAPABILITIES
-------------------------------
HPM.1 version............0
Component 0 presence....[y]
Component 1 presence....[y]
Component 2 presence....[y]
Component 3 presence....[y]
Component 4 presence....[n]
Component 5 presence....[n]
Component 6 presence....[n]
Component 7 presence....[y]
Upgrade undesirable.....[n]
Aut rollback override...[n]
IPMC degraded...........[y]
Defered activation......[n]
Service affected........[n]
Manual rollback.........[n]
Automatic rollback......[n]
Self test...............[n]
Upgrade timeout.........[300 sec]
Self test timeout.......[0 sec]
Rollback timeout........[300 sec]
Inaccessibility timeout.[300 sec]

GENERAL PROPERTIES
-------------------------------
Payload cold reset req....[n]
Def. activation supported.[n]
Comparison supported......[n]
Preparation supported.....[y]
Rollback supported........[y]

Description string: CMC FW

Current Version:
 Major: 00
 Minor: 0b
 Aux  : 00 00 00 00

Validating firmware image integrity...OK
Performing preparation stage...TARGET UPGRADE CAPABILITIES
-------------------------------
HPM.1 version............0
Component 0 presence....[y]
Component 1 presence....[y]
Component 2 presence....[y]
Component 3 presence....[y]
Component 4 presence....[n]
Component 5 presence....[n]
Component 6 presence....[n]
Component 7 presence....[y]
Upgrade undesirable.....[n]
Aut rollback override...[n]
IPMC degraded...........[y]
Defered activation......[n]
Service affected........[n]
Manual rollback.........[n]
Automatic rollback......[n]
Self test...............[n]
Upgrade timeout.........[300 sec]
Self test timeout.......[0 sec]
Rollback timeout........[300 sec]
Inaccessibility timeout.[300 sec]

GENERAL PROPERTIES
-------------------------------
Payload cold reset req....[n]
Def. activation supported.[n]
Comparison supported......[n]
Preparation supported.....[y]
Rollback supported........[y]

OK
Error initiating upgrade action, compcode = ff

Firmware upgrade procedure failed

Error aborting upgrade, compcode = 80

...

-----------------------------------------------------------------
|ID | Name      | Active  |   Upload Progress   | Upload| Image |
|   |           | Version |0%      50%      100%| Time  | Size  |
|---|-----------|---------||----+----+----+----||-------|-------|
...Failed...
-----------------------------------------------------------------

In OneFS 8.0, this would also cause output similar to the following for upgrade commands:

Isilon-1# isi upgrade nodes firmware progress list
Lnns  Device               Old Version             New Version             Status
-----------------------------------------------------------------------------------
...
6     CMCSDR_Yeti          00.06                   00.06                   -
6     CMC_Yeti             00.0b                   02.05                   error
6     IsilonFPV1           UI.01.36                UI.01.36                -
...
-----------------------------------------------------------------------------------
Total: 75
Isilon-1# isi upgrade nodes view 6
             Node LNN: 6
   Node Upgrade State: error
        Error Details: Failed Step: InstallFirmware
                       Error Log: /ifs/.ifsvar/upgrade/Agent-6-000e1e591a20232c8a58011809d7e56ff3fb.log
  Last Upgrade Action: -
   Last Action Result: -
Node Upgrade Progress: None
         Node OS Version: 8.0.0.4_build(53)style(5)

Isilon-1# isi_upgrade_logs
Copied log to /ifs/upgrade_error_logs/6_isi_Upgrade_Agent.py-InstallFirmware.log

Isilon-1# cat /ifs/upgrade_error_logs/6_isi_Upgrade_Agent.py-InstallFirmware.log
Starting /usr/sbin/isi_Upgrade_Agent.py InstallFirmware

Verifying boot device mirrors are synced...
2017-05-11 15:18:42,876 DEBUG Execute Command: InstallFirmware
2017-05-11 15:18:59,005 ERROR Failed to update CMC_Yeti firmware on Node 6

Cause

이 문제는 노드 펌웨어 업데이트 중에 CMC 구성 요소가 적시에 응답하지 않아 발생합니다.

Resolution

이 문제는 노드에서 BMC 및 CMC 구성 요소의 콜드 재설정을 수행하여 해결할 수 있습니다. 콜드 리셋으로 문제가 해결되지 않을 경우 https://support.emc.com/kb/466373 에 따라 노드의 전원을 끄고 전기를 방전시켜야 합니다.  어떤 단계로도 문제가 해결되지 않으면 추가 분석을 위해 Isilon 기술 지원 팀으로 에스컬레이션하십시오.

CMC를 재설정하려면 영향을 받는 노드에서 /usr/bin/isi_hwtools/isi_ipmicmc -c -a cmc

명령을 실행합니다.
예제 출력: 
Isilon-6# /usr/bin/isi_hwtools/isi_ipmicmc -c -a cmc
Cold reset CMC
Note - 15 sec wait for CMC...
Checking if Firmware Version MSB is set
MSB set in Firmware Version
Waiting 5 seconds before trying again
Success: MSB cleared in Firmware Version
Checking self test results
Success: self test passed

Once reset, retry the firmware update. 
In OneFS versions prior to 8.0:
isi firmware update

In OneFS versions 8.0 and later:
Review the 'Node Upgrade State' for each node, identify the node that is in an error state, this is the node that the upgrade will be retried on:
isi upgrade nodes list

To retry the upgrade on the node in an error state, run the following command, replace LNN with the node that is showing in an error state:
isi upgrade retry-last-action LNN
 

Affected Products

Isilon

Products

Isilon, Isilon HD400, Isilon NL410, Isilon S210, Isilon X210, Isilon X410
Article Properties
Article Number: 000050382
Article Type: Solution
Last Modified: 11 Oct 2024
Version:  3
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.