Connectrix Brocade: Firmware download failed on standby CP - Firmware download timed out (0x23)
Summary: FOS upgrade from v8.2.1c to v8.2.2d failed due to "Firmware download failed on standby CP - Firmware download timed out. (0x23)
This article applies to
This article does not apply to
This article is not tied to any specific product.
Not all product versions are identified in this article.
Symptoms
Loopback communication between the CPs was observed during the upgrade of the switch.
During the firmwaredownload on the CPs, both CPs were rebooted and regained the communications on the backend.
Increasing the timeout value of the firmwaredownload command with the example shown below didn't help.
In this example, it increases the timeout from the default of 30 minutes (1800 seconds) to 40 minutes (2400 seconds).
During the firmwaredownload on the CPs, both CPs were rebooted and regained the communications on the backend.
Apr 6 01:22:42 softupgrade_main:CP1 ### Firmwaredownload command has started ###
Apr 6 01:22:42 softupgrade_main:CP1 IpcInit failed. Firmwaredownloadis already running.
Apr 6 01:22:42 set_status:CP1 Firmwaredownload failed because already another instance of firmwaredownload operation is in progress. Another firmwaredownload is in progress. (0x18)
Apr 6 01:28:06 ipc_send_recv:CP1 err = 35, error_code = 35
Apr 6 01:28:25 set_status:CP1 Firmware download failed on standby CP - Firmware download timed out. (0x23)
Apr 6 01:28:25 state_fwdl_begin:CP1 Firmwaredownload status in state_trans(): Firmware download timed out.
Apr 6 01:28:30 sus_notify:CP1 Firmwaredownload failed. Calling ismic_send_fwdnld_fail.
Apr 6 01:28:31 sus_notify:CP1 Status is SUS_NOTIFY_FWDL_FAILED
Apr 6 01:28:33 ping_standby:CP1 ping_standby(): cmd = /bin/ping -q 127.3.1.1 -c 2 -i 2 1>/dev/null, ret=0x0
Apr 6 01:28:33 ping_standby:CP1 ping_standby failed but htbt ok. Ignore
Apr 6 01:29:22 fwdl_parse_cmd_option:CP1 option=0, index = 1
Apr 6 01:29:54 set_status:CP1 Firmwarecommit has started on the remote CP. Please use firmwaredownloadstatus and firmwareshow to see the firmware status.
Apr 6 01:29:54 set_status:CP1 Please use firmwaredownloadstatus and firmwareshow to see the firmware status.
Apr 6 01:29:54 state_fwdl_begin:CP1 Returning SUS_PEER_FWDL_FAIL
Mar 23 22:37:16 set_status:CP1 Firmware download failed on standby CP - Firmware download timed out. (0x23)
Mar 23 22:37:16 state_fwdl_begin:CP1 Firmwaredownload status in state_trans(): Firmware download timed out.
Mar 23 22:37:21 sus_notify:CP1 Firmwaredownload failed. Calling ismic_send_fwdnld_fail.
Mar 23 22:37:21 sus_notify:CP1 Status is SUS_NOTIFY_FWDL_FAILED
Mar 23 22:37:23 ping_standby:CP1 ping_standby(): cmd = /bin/ping -q 127.3.1.1 -c 2 -i 2 1>/dev/null, ret=0x0
Mar 23 22:37:23 ping_standby:CP1 ping_standby failed but htbt ok. Ignore
Mar 23 22:38:44 set_status:CP1 Firmwarecommit has started on the remote CP. Please use firmwaredownloadstatus and firmwareshow to see the firmware status.
Mar 23 22:38:45 set_status:CP1 Please use firmwaredownloadstatus and firmwareshow to see the firmware status.
Mar 23 22:38:45 state_fwdl_begin:CP1 Returning SUS_PEER_FWDL_FAIL
Apr 6 01:22:42 softupgrade_main:CP1 IpcInit failed. Firmwaredownloadis already running.
Apr 6 01:22:42 set_status:CP1 Firmwaredownload failed because already another instance of firmwaredownload operation is in progress. Another firmwaredownload is in progress. (0x18)
Apr 6 01:28:06 ipc_send_recv:CP1 err = 35, error_code = 35
Apr 6 01:28:25 set_status:CP1 Firmware download failed on standby CP - Firmware download timed out. (0x23)
Apr 6 01:28:25 state_fwdl_begin:CP1 Firmwaredownload status in state_trans(): Firmware download timed out.
Apr 6 01:28:30 sus_notify:CP1 Firmwaredownload failed. Calling ismic_send_fwdnld_fail.
Apr 6 01:28:31 sus_notify:CP1 Status is SUS_NOTIFY_FWDL_FAILED
Apr 6 01:28:33 ping_standby:CP1 ping_standby(): cmd = /bin/ping -q 127.3.1.1 -c 2 -i 2 1>/dev/null, ret=0x0
Apr 6 01:28:33 ping_standby:CP1 ping_standby failed but htbt ok. Ignore
Apr 6 01:29:22 fwdl_parse_cmd_option:CP1 option=0, index = 1
Apr 6 01:29:54 set_status:CP1 Firmwarecommit has started on the remote CP. Please use firmwaredownloadstatus and firmwareshow to see the firmware status.
Apr 6 01:29:54 set_status:CP1 Please use firmwaredownloadstatus and firmwareshow to see the firmware status.
Apr 6 01:29:54 state_fwdl_begin:CP1 Returning SUS_PEER_FWDL_FAIL
Mar 23 22:37:16 set_status:CP1 Firmware download failed on standby CP - Firmware download timed out. (0x23)
Mar 23 22:37:16 state_fwdl_begin:CP1 Firmwaredownload status in state_trans(): Firmware download timed out.
Mar 23 22:37:21 sus_notify:CP1 Firmwaredownload failed. Calling ismic_send_fwdnld_fail.
Mar 23 22:37:21 sus_notify:CP1 Status is SUS_NOTIFY_FWDL_FAILED
Mar 23 22:37:23 ping_standby:CP1 ping_standby(): cmd = /bin/ping -q 127.3.1.1 -c 2 -i 2 1>/dev/null, ret=0x0
Mar 23 22:37:23 ping_standby:CP1 ping_standby failed but htbt ok. Ignore
Mar 23 22:38:44 set_status:CP1 Firmwarecommit has started on the remote CP. Please use firmwaredownloadstatus and firmwareshow to see the firmware status.
Mar 23 22:38:45 set_status:CP1 Please use firmwaredownloadstatus and firmwareshow to see the firmware status.
Mar 23 22:38:45 state_fwdl_begin:CP1 Returning SUS_PEER_FWDL_FAIL
Increasing the timeout value of the firmwaredownload command with the example shown below didn't help.
#firmwaredownload -x 2400
In this example, it increases the timeout from the default of 30 minutes (1800 seconds) to 40 minutes (2400 seconds).
Cause
Loopback communication issues between the CPs
Resolution
Upgrade the switch CP's using the #firmwaredownload -s option which is a single CP controller upgrade
- Use the command #firmwaredownload -s to download the firmware to standby CP and perform #hafailover to reboot it.
- Perform the same on other CP.
Additional Information
"Firmware download timed out. (0x23)" is not a bug as it is usually due to the timing issues between the Ethernet port of the Brocade Switch and the SFTP/FTP/SCP Server.
Therefore, by using the "firmwaredownload -s" reduces the time needed to coordinate the file transfer from the Active CP to the Standby CP when the normal "firmwaredownlaod" command is used (as the command will start the firmware upgrade with the "Standby" CP first).
This kind of issues in the past was resolved by checking the Ethernet port settings, extended the timeout value, or in this case the customer used "firmwaredownload -s" command on each CP to bypass the additional time needed for the communications between the CPs.
As these CPs had already been rebooted during the firmwaredownload, this issue shouldn't be re-occurring on the next upgrade.
Therefore, by using the "firmwaredownload -s" reduces the time needed to coordinate the file transfer from the Active CP to the Standby CP when the normal "firmwaredownlaod" command is used (as the command will start the firmware upgrade with the "Standby" CP first).
This kind of issues in the past was resolved by checking the Ethernet port settings, extended the timeout value, or in this case the customer used "firmwaredownload -s" command on each CP to bypass the additional time needed for the communications between the CPs.
As these CPs had already been rebooted during the firmwaredownload, this issue shouldn't be re-occurring on the next upgrade.
Affected Products
BrocadeArticle Properties
Article Number: 000185590
Article Type: Solution
Last Modified: 01 Aug 2021
Version: 3
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.