Highlighted
nickw-
Bronze

SSH Sessions repeatedly hang on 3348.

There appears to be a bug in the 1.0.0.106 firmware. I've repeated it on more than one identical switch.

Sometimes when doing a "show spanning-tree" the session will drop/hang open on the switch.

After this has happened once, and you've reconnected, many other commands will make it freeze (usually immediately) just pressing '?' will hang it.

Of course after having 5 sessions hung, you can't connect again.


excerpts from a log I made follow:

s1-c8# show span


Spanning tree enabled mode MSTP
Default port cost method: short
... etc...

/***Typed the same command several more times***/

s1-c8# show span


Spanning tree enabled mode MSTP
Default port cost method: short


Gathering information ..........Read from remote host s1-c8: Connection reset by peer
Connection to s1-c8 closed.

nick@gorf:~$ ssh s1-c8

01-Jan-2000 01:14:28 %AAA-I-CONNECT: User CLI session for user unKnown over ssh , source 204.174.223.110
destination 10.100.32.71 ACCEPTED
01-Jan-2000 01:14:28 %AAA-I-CONNECT: User CLI session for user unKnown over ssh , source 204.174.223.110
destination 10.100.32.71 ACCEPTED

s1-c8> en
Password:********
01-Jan-2000 01:15:01 %AAA-I-CONNECT: User CLI session for user unKnown over ssh , source 204.174.223.110
destination 10.100.32.71 ACCEPTED
01-Jan-2000 01:15:01 %AAA-I-CONNECT: User CLI session for user unKnown over ssh , source 204.174.223.110
destination 10.100.32.71 ACCEPTED

s1-c8# show span /*** HUNG ***/



*** same thing a couple more times ***

nick@gorf:~$ ssh s1-c8


01-Jan-2000 01:15:59 %AAA-I-CONNECT: User CLI session for user unKnown over ssh , source 204.174.223.110
destination 10.100.32.71 ACCEPTED
01-Jan-2000 01:15:59 %AAA-I-CONNECT: User CLI session for user unKnown over ssh , source 204.174.223.110
destination 10.100.32.71 ACCEPTED
01-Jan-2000 01:15:59 %AAA-I-CONNECT: User CLI session for user unKnown over ssh , source 204.174.223.110
destination 10.100.32.71 ACCEPTED

s1-c8> /*** This time I just pressed ? and it hung ***/

nick@gorf:~$ ssh s1-c8
Connection to s1-c8 closed.
nick@gorf:~$ ssh s1-c8
Connection to s1-c8 closed.
nick@gorf:~$

/*** Now there's no more free sessions ***/
0 Kudos
13 Replies
DELL-Cuong N.
Platinum

Re: SSH Sessions repeatedly hang on 3348.

I'm not familiar with version 1.0.0.106 on a 3348.  Do you mean 3448 or really 3348?  If you are using 3348 then that FW looks to be very old.  You might consider upgrading to the latest version 1.2.0.6 (go to support.dell.com).  Also there is a newer version of the 3448 FW too.  The latest version for 3448 is 1.0.0.112.
 
I have not seen this problem before and neither has my colleagues.  You might try upgrading to the latest FW to see if it helps.
 
Cuong.
0 Kudos
nickw-
Bronze

Re: SSH Sessions repeatedly hang on 3448.

Sorry, you're right. It is a 3448.
0 Kudos
nickw-
Bronze

Re: SSH Sessions repeatedly hang on 3348.

Still does the same thing with 112
0 Kudos
DELL-Cuong N.
Platinum

Re: SSH Sessions repeatedly hang on 3348.

Does it happens only on an SSH session or does it happens also when you simply telnet into the switch?  Also does it happens only when you have all five SSH sessions or can it happens when only one SSH session is active.  How easily is it to reproduce?  Do you have to do "show spanning" repeatedly over many times to reproduce it?

Any info that might help me reproduce the problem is appreciated.

0 Kudos
nickw-
Bronze

Re: SSH Sessions repeatedly hang on 3448.

It happened to me while using SSH, though I haven't tested it with either the serial console or telnet.

Reproducing it is quite easy.

It doesn't seem to be consistent in the number of times you have to show spanning-tree. The first time it happened was while configuring some spanning tree settings, so in that case, it was only run a few times between other commands. When I first tried to reproduce it it took several tries (maybe 5-6), though sometimes it may take 20 or 30. You should be able to reproduce it easily by just repeating the command many times.

If you don't have any luck with that I can send you my config, though I don't know that that will be necessary.
0 Kudos
DELL-Cuong N.
Platinum

Re: SSH Sessions repeatedly hang on 3448.

I have tried with both SSH and regular telnet sessions.  I have executed the show spanning tree operation more then 50 times on each type of sessions.  I also tried with multiple telnet & SSH sessions active.  I'm using SSH on cygwin and on Linux.  No luck reproducing it.

Can you give me more information please.  When you said it locked up on you, does it hangs only the given SSH session so that you can access it with another SSH session?  If so maybe something to do with your particular SSH tool.  What are you using?  Is it SSH that comes with Linux or something on windows?

Does the console show any message when a session "locks up" on you?  Any other observation that might help?

Cuong.

0 Kudos
DELL-Cuong N.
Platinum

Re: SSH Sessions repeatedly hang on 3448.

I think I figured out the problem.  It seems that some SSH clients on windows will seem to lock up when use with the switch.  It maybe that the performance on these clients are so slow that they look as if they lock up when executing commands that result in a lot of data being returned (such as "show spanning").

I have downloaded and tried a number of different SSH clients on Windows and I find that there are really three types:

  • SSH clients based on OpenSSH which requires cygwin - this is what I used normally and these works fine.
  • SSH clients based on Putty.  These seems to have some performance problems when connected to the switch.  There are a number of these types of clients all based on Putty including the one in "TTermpro" (look here for some information on these types of clients: http://www.openssh.com/windows.html).
  • Alternatively you can also use SSHClient which is not based on OpenSSH or Putty and seems to have the best performance of all SSH clients available on Windows.  See here for information on this client: http://ecs.rutgers.edu/ssh_windows.php3.  The latest version of this client is SSHSecureShellClient version 3.2.9 found here (ftp://sunsite.unc.edu/pub/packages/security/ssh/).

Also when these slower clients are killed they do not release the session which cause the problems you notice where the sessions are locked open.  In fact if you are really patience these clients do eventually response just take a VERY long time.

I suggest you use the OpenSSH based clients on windows or the SSHSecureShellClient for better performance.

Cuong.

0 Kudos
nickw-
Bronze

Re: SSH Sessions repeatedly hang on 3448.

I have been using OpenSSH in linux on more than one seperate machine.

"OpenSSH_3.8.1p1 Debian-8.sarge.4, OpenSSL 0.9.7g 11 Apr 2005"


The first time the problem occurs the session disconnects "Connection reset by peer" as in the log in my original message.

Subsequent attempts to do show span (or many other commands) will hang the session. (no response within several minutes) the session seems to have hung completely and stays open on the switch even after killing my client and connecting again.

Here's the config, incase that's helpful.

As I mentioned before, I've tried it on 2 switches with the this config, and both had the same problem.




nick@gorf:~$ ssh s1-c8


s1-c8> en
Password:********

s1-c8# show running-config
spanning-tree mode mstp
interface range ethernet g(3-4)
switchport mode trunk
exit
vlan database
vlan 30,50,70
exit
interface range ethernet e(1-48)
switchport access vlan 50
exit
interface range ethernet g(3-4)
switchport trunk allowed vlan add 50
exit
interface range ethernet g(3-4)
switchport trunk allowed vlan add 70
exit
interface vlan 30
name colo
exit
interface vlan 50
name colo2
exit
More: , Quit: q, One line: interface vlan 70
name management
exit
interface vlan 70
ip address 10.100.32.71 255.255.255.0
exit
ip default-gateway 10.100.32.254
hostname s1-c8
line console
autobaud
exit
line console
speed 115200
exit
aaa authentication login default none
line ssh
password 00000000000000000000000000000000 encrypted
exit
enable password level 15 00000000000000000000000000000000 encrypted
username \ password 00000000000000000000000000000000 encrypted
ip ssh server
snmp-server community Dell_Network_Manager rw view DefaultSuper
More: , Quit: q, One line:





Default settings:
Service tag: GYWYK71

SW version 1.0.0.106 (date 01-Jun-2005 time 15:26:21)

Fast Ethernet Ports
==========================
no shutdown
speed 100
duplex full
negotiation
flow-control off
mdix auto
no back-pressure

Gigabit Ethernet Ports
More: , Quit: q, One line: =============================
no shutdown
speed 1000
duplex full
negotiation
flow-control off
mdix auto
no back-pressure

interface vlan 1
interface port-channel 1 - 8

spanning-tree
spanning-tree mode STP

qos basic
gorf%
0 Kudos
DELL-Cuong N.
Platinum

Re: SSH Sessions repeatedly hang on 3448.

Ok.  Let me try this config on my switch and I will let you know.  However, I have been unable to reproduce your problem on my current system using various different SSH implementation on Linux including OpenSSH.  Maybe its related to your configuration.  Anyway, I will load your config and try it.

Also, have you tried it with just telnet just to eliminate the SSH issue?

Cuong.

0 Kudos