Symptoms
Impact:
Loss of functionality of the switch
Switch not coming back up after reboot.
Environment:
Dell Hardware: Connectrix DS-6610B (Only DS-6610B are at risk.)
Brocade Software: Fabric OS 9.1.0
Brocade Software: Fabric OS 9.1.1
Problem:
Upgrading switch from Fabric OS 9.1.0x to any version of Fabric OS 9.1.1x or Fabric OS 9.2.0 could be at risk to fail to boot up. When there is a power outage, or there the power cable is pulled.
The DS-6610b switch must have at one time being loaded with a version of FOS v9.1.0x to be at risk, to encounter this problem.
The same DS-6610B switch must then have been upgraded to any version of FOS v9.1.1x (v9.1.1_01, v9.1.1a, v9.1.1b), or to FOS v9.2.0 and not upgraded again after that, remaining on this new code level.
Console output shows: "ERROR: can't get kernel image!"
Serial console output example:
- Vendor: 0x1c79 Product 0x3100 Version 1.0
2 USB Device(s) found
scanning usb for storage devices... 1 Storage Device(s) found
setting prt to 2
Secure Boot Mode Enabled, System in Trusted State
Hit ESC to stop autoboot: 0
** File not found /boot/signed_bzImage.itb **
2805 bytes read in 430 ms (5.9 KiB/s)
## Executing script at 00200000
sha1,rsa2048:dev+ sha1+ WARNING: adjusting available memory to 30000000
Wrong Image Format for bootm command
ERROR: can't get kernel image!
For this problem to exist, the DS-6610b switch must have at one time been loaded with a version of FOS v9.1.0x to be at risk.
The same DS-6610B switch must then have been upgraded to any version of FOS v9.1.1x (v9.1.1_01, v9.1.1a, v9.1.1b) or to FOS v9.2.0 and not upgraded again after that, remaining on this new code level.
Finally, the switch must be powered off while still at this firmware version.
The following upgrade scenarios would leave a DS-6610B at risk of encountering this failure:
- FOS v9.0.1x => v9.1.0 or v9.1.0b to v9.1.1_01 and then powered off while at this code level
- FOS v9.0.1x => v9.1.0 or v9.1.0b to v9.1.1a and then powered off while at this code level
- FOS v9.0.1x => v9.1.0 or v9.1.0b to v9.1.1b and then powered off while at this code level
- FOS v9.0.1x => v9.1.0 or v9.1.0b to v9.2.0 and then powered off while at this code level
Switches that are upgraded from v9.1.1x to a higher FOS version or that are upgraded directly from v9.0.1x to v9.1.1x are NOT at risk as shown in the following upgrade path examples:
- FOS v9.0.1x => v9.1.0 or v9.1.0b => v9.1.1_01 to v9.1.1b Not at risk to power off issue
- FOS v9.0.1x => v9.1.0 or v9.1.0b => v9.1.1a to v9.1.1b Not at risk to power off issue
- FOS v9.0.1x => v9.1.0 or v9.1.0b => v9.1.1b to v9.2.0 Not at risk to power off issue
- FOS v9.0.1x => v9.1.1_01 Not at risk to power off issue
- FOS v9.0.1x => v9.1.1a Not at risk to power off issue
- FOS v9.0.1x => v9.1.1b Not at risk to power off issue
Switches that were shipped originally loaded with v9.1.1a or v9.1.1b, and never downgraded to a v9.1.0x FOS version, are also not at risk to encounter this issue.
The failure is only encountered following a power cycle.
- Switch reboot does not cause the failure
- HA reboot does not cause the failure
DS-6610B switches with Lifetime Warranty are not at risk, as only the v9.0.1e1_LW and v9.1.1b_LW FOS versions were ever posted for download.
FOS v9.1.0b_LW was never posted for download onto a DS-6610B-LW switch, and DS-6610B-LW switches never having been loaded with a FOS v9.1.0x FOS version will not be at risk.
Cause
Brocade DS-6610B running FOS v9.1.1 has a DEFECT in the code, where the boot images cannot be found. Causing the switch not able to reboot after power outage or power cycle.
And it is specific the DS-6610B, and a fix has been created and implemented in newer Fabric OS v9.1.1a.
Starting with v9.1.1, during post-installation of G6_ENTRY code, symbolic links of Linux image files are used on the G610 platform.
The name of one of the symbolic links created has the same name as the Linux security boot image rpm file created during FOS v9.1.0x.
After upgrading from v9.1.0x to v9.1.1x/v9.2.0, the previously installed boot file is deleted as the new symbolic link file was created. However, since the symbolic link has the same name, it is also deleted by mistake.
This failure is not detected due to a defect in the u-boot code.
The symbolic link file will be recreated on any future upgrade, but while the G610 remains at a FOS version with the file deleted, it is unable to boot following a power failure or power cycle.
HA events and reboots are not affected by this missing file.
Brocade DEFECT FOS-844483
Brocade TSB 2023-290-A
Resolution
FIX:
When issue encountered, replace (RMA) the switch.
Workaround:
For DS-6610B running any v9.0.1x version of FOS, upgrade directly to one of the 9.1.1.x versions.
For DS-6610B running any versions of v9.1.0x, only upgrade to 9.1.1c tor 9.2.0a or higher versions.
For DS-6610B running v9.1.1x or FOS v9.2.0 and that had been previously loaded with a version of FOS v9.1.0x prior to this single time upgrade, upgrade to higher version of FOS.
- The v9.1.0x file has already been deleted, and the symbolic link file is recreated on this second upgrade.)
- Or optionally perform a firmwarecleaninstall to the same version of v9.1.1x or v9.2.0 FOS that is running. This restores the missing file, and avoids any issue due to a power event. Refer to documentation manuals for the steps and procedure required to perform a firmware clean install.
Upgrade to v9.1.1c or v9.2.0a or higher version of FOS with the fix.