Unsolved
This post is more than 5 years old
5 Posts
0
48186
8024F Upgrade from 3.x.x to 4.x.x
All,
I've been having some problems upgrading my 8024F to version 4 firmware. The main issue is that ftp (and thus TFTP) transfer to the switch is out due to security concerns. Http transfer works fine, until you upgrade to the interim firmware (vA.0.1.19) which for some reason doesn't let HTTP work (it gives a 404 when you reach the page), and since the upgrade requires that you go to the interim firmware before moving to the 4.x firmware, i'm a little stuck. My transfer options are: FTP/TFTP/SFTP (out), HTTP (404), XMODEM (fun), and SCP.
I've gotten ssh working just fine, even on the interim firmware. No problems there. but for some reason, whenever i try to transfer the new firmware to the switch via SCP, it completely hangs the management interface! No matter how long i wait, the management interface doesn't come back from the transfer (waited ~22 hours) and you can't cancel or otherwise interrupt the transfer. Resetting the switch brings it back up, but we still didn't transfer over the firmware. I'm down to my last option here, so i'm gonna hafta give xmodem a try. Shouldn't be a problem, but i still would like to know what's up with HTTP and SCP? Any thoughts?
seventhsven
19 Posts
0
September 20th, 2011 05:00
We've run into this issue numerous times with at least firmware 3.1.3.9 and the latest 3.1.5.13. It seems to be an obvious bug in the v3 firmware. I prepared a bug report for Dell support, but haven't sent it yet. Hopefully it will get past their first line without a quick "use firmware v4" response. And yes, v4 (tested on 4.1.0.19) seems to work fine with scp/sftp.
Here's the report. Attachments won't be available, of course. They are described, though.
# Start of report
Bug report for Dell PowerConnect M6220.
System: PowerConnect M6220, 3.1.5.13, VxWorks 6.5
Problem summary:
Using scp or sftp to upload files from the switch CLI or web
interface, hangs the switch management indefinitely.
Steps to reproduce:
Power up factory-fresh Dell M1000e with one M6220 in slot A1. For
completeness, this particular system was also set up with a M6220 in
slot A2 and 16 M610 blades (powered down). No stacking between the
M6220s.
Apply a static IP address on switch-1 using the CMC command:
racadm setniccfg -m switch-1 -s x.x.x.2 255.255.255.0 x.x.x.1
Upgrade the firmware on switch-1 from the factory default 3.1.3.9 to
3.1.5.13 using the web interface. HTTP transfer method. Reboot the
switch with the 3.1.5.13 image set to activate on next boot.
Using either the web interface or the CLI, start an SCP or SFTP upload
of any file to an external host. Here's how it looks in the CLI:
====START====
console#copy startup-config scp://testuser@x.x.x.3/foo.cfg
Remote Password:******
Mode........................................... SCP
Set TFTP Server IP............................. x.x.x.3
TFTP Path...................................... ./
TFTP Filename.................................. foo.cfg
Data Type...................................... Config Script
Source Filename................................ startup-config
Management access will be blocked for the duration of the transfer
Are you sure you want to start? (y/n) y
SCP Configuration Script transfer starting...
0 bytes transferred...
====END====
What seems to happen at this point, is that the switch connects to the
remote host, then starts to do the SSH protocol handshake. Once the
remote host has sent SSH2_MSG_KEXINIT, the switch should also send its
SSH2_MSG_KEXINIT back to the host. Instead, the switch goes into an
infinite loop sending empty packets.
At this point, the CLI becomes unresponsive. The web interface lets
you connect to it, but it returns the scp/sftp progress html in all
frames, rendering the web ui useless and impossible to log in to.
To recover from this situation, the only solution so far seems to be
a switch reset through the CMC: "chassisaction -m switch-1 reset"
Attachments and their descriptions:
m6220_scpfreeze__show_tech-support.txt: Tech support output from
switch, after firmware upgrade, before copy command is run.
m6220_scpfreeze__copy_command.txt: Text output of scp command that
fails. The console and web ui freeze right after this output.
m6220_scpfreeze__postfreeze_loginscreen.png: Screenshot of the login
screen after the freeze has occurred. Notice how every frame and every
object on the page is replaced by the scp/sftp progress html.
m6220_scpfreeze__packet_capture.pcap: Packet capture showing how the
switch and remote host do the SSH handshake. The switch fails to
complete the key exchange, and instead goes into an infinite packet
loop. The capture file is cut short to avoid a large file size.
m6220_scpfreeze__sshd_debug.txt: Host-side sshd debug output.
Corresponds with the packet capture, and shows how the handshake
stalls after the host sends the key exchange init.
m6220_scpfreeze__v3.1.3.9_stacktrace.txt: A similar but older
occurrence of the same problem on firmware 3.1.3.9. This one might
prove interesting, as it has a stack trace in the output. It is
unclear if this 3.1.3.9 crash caused the packet loop we see with
3.1.5.13. The switch hung, and failed over its stack manager role to
the other switch in the stack.
# End of report
jp1110
67 Posts
0
September 25th, 2011 09:00
Are you attempting to do this remotely or do you have physical access to the swtich?
seventhsven
19 Posts
0
September 25th, 2011 14:00
It's done by logging in via ssh to the m1000e cmc, then connecting to the oob switch management console.
On a side note, I reported this issue a few days ago to Dell support. I'll post updates here as I get info from them.
_JoelW
5 Posts
0
September 26th, 2011 07:00
This is absolutely fantastic, thank you for going to all the trouble of tracking down this bug. I was beginning to worry that I was suffering from some sort of specific, Dell hating delusion. :)
_JoelW
5 Posts
0
September 26th, 2011 12:00
Although, i wanted to point out that I am still having scp problem as of version 4.1.0.19. The behavior is identical to version 3.x. I haven't packet sniffed or sshd debugged it as i've been fairly busy, but i'll give it a try when i get some time.
jp1110
67 Posts
0
September 26th, 2011 18:00
Yup understand that... Can you log into your switch and do a show run. Make sure your not running simple switch mode? There are many features that have been removed from the Full OS to the simple switch mode...
seventhsven
19 Posts
0
September 27th, 2011 01:00
So you still see the problem on 4.1.0.19. That's very interesting, and I bet Dell support are interested too. Could you do the packet dumping/testing/debugging to see what might go wrong?
Here's the answer from Dell support, where the conclusion is that v3 won't see any changes or bug fixes at all:
===QUOTE START===
Version 3.x.x.x of software for all blade (M6220, M6348) switches as well as PC70xx and PC80xx series switches is in the END OF DEVELOPMENT status.
All of these switches have v.4.1.x.x available and this is the only version which will be developed.
Long story short there are no further plans to release any v.3.x.x.x even for bug fixes.
===QUOTE END===
jp1110
67 Posts
0
October 11th, 2011 00:00
Update?
arobb
1 Message
0
February 12th, 2012 16:00