Unsolved
5 Posts
0
842
PowerConnect 6248 PCT6248 stops forwarding traffic intermittently
Management on VLAN 1 and user traffic on VLAN 1 stops. Although I have VLANs configured, I am only using VLAN 1.
The log on SWITCH_119A shows nothing. I only have three user connections on this switch, ports 13, 16, & 44. Through trial and error, I have that if I bounce port 44 (SHUT/NO SHUT), the switch immediately begins transmitting traffic. The failure is sporadic, a few times per week. Sometimes during the day, other times in the early morning hours when no users are in.
I have an identical switch SWITCH_119B in the same rack. I added SWITCH_119B to the same rack and with its own uplink to the core for troubleshooting. When SWITCH_119A would fail, I would move my patch cables from switch A to B on the corresponding port. When doing this, I discovered that when port 44 was disconnected, "A" would heal.
When I left my users on "B", it experienced the same failure. Disconnecting (or SHUT/NO SUT) port 44 would allow it to heal.
I now have all but 3 connections on switch "B" and it is not failing.
Port 44 has a docking station that allows various Dell laptops to connect.
Switch "A" has the remaining 3 connections.
When switch "A" fails, since it is isolated, it comes the spanning-tree root for itself. This caused the CoreSwitch to disable the downlink port 47 to switch "A" as a spanning-tree discard state. Thus, it appears BPDU is passing but not production traffic. I disabled spanning-tree on the CoreSwitch downlink port 47 to switch "A" to eliminate a possible spanning-tree issue.
~~~~~~~~~~~~~~~~~~~~~~~
SWITCH_119A#show system
System Description: Dell Ethernet Switch
System Up Time: 7 days, 19h:02m:08s
System Contact:
System Name: SWITCH_119A
System Location:
Burned In MAC Address: D067.E5A3.6BDF
System Object ID: 1.3.6.1.4.1.674.10895.3013
System Model ID: PCT6248P
Machine Type: PowerConnect 6248P
SWITCH_119A#show running-config interface ethernet 1/g47
switchport mode general
switchport general allowed vlan 63,110,200 tagged
lldp transmit-tlv port-desc sys-name sys-desc sys-cap
lldp transmit-mgt
SWITCH_119A#show isdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge,
S - Switch, H - Host, I - IGMP, r - Repeater
Device ID Intf Holdtime Capability Platform Port ID
---------------------- ------- --------- ------------ -------------------- ----------------------
CoreSwitch 1/g48 145 R N2048 Gi1/0/47
~~~~~~~~~~~~~~~~~~~~~~~
CoreSwitch#show system
System Description: Dell Networking Switch
System Up Time: 55 days, 22h:54m:06s
System Contact: Network_Admin
System Name: CoreSwitch
System Location: Room 223
Burned In MAC Address: F8B1.5654.8321
System Object ID: 1.3.6.1.4.1.674.10895.3054
System Model ID: N2048
Machine Type: Dell Networking N2048
CoreSwitch#show isdp neigh
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge,
S - Switch, H - Host, I - IGMP, r - Repeater
Device ID Intf Holdtime Capability Platform Port ID
------------------------ --------- --------- ---------- ---------------- ---------
SWITCH_119B Gi1/0/4 155 R PCT6248P 1/0/48
SWITCH_Rm113 Gi1/0/7 179 R PCT6248P 1/0/48
SWITCH_Rm207 Gi1/0/21 156 R PCT6248P 1/0/48
PHSWITCH1 Gi1/0/40 165 R PCT6248P 1/0/40
Switch98B9D1 Gi1/0/42 143 S I Cisco SG220-28MP gi25
Switch98BBA1 Gi1/0/42 155 S I Cisco SG220-28MP gi25
Switch98BF41 Gi1/0/42 118 S I Cisco SG220-28MP gi25
SWITCH2_Rm223 Gi1/0/46 157 R PCT6248P 1/0/44
SWITCH_119A Gi1/0/47 154 R PCT6248P 1/0/48
SWITCH3_Rm223 Gi1/0/48 160 R PCT6248P 1/0/48
SWITCH_WestBldg Te1/0/1 160 R PCT6248P 1/0/45
CoreSwitch#show running-config interface gi1/0/47
description "SWITCH119A"
spanning-tree disable
switchport mode general
switchport general allowed vlan 63,110,200 tagged
DELL-Charles R
Moderator
Moderator
•
3.4K Posts
0
October 11th, 2022 13:00
Hello GregT_at_FCSG,
I would like to gather some more information.
Can you provide a topology layout?
Also run this and provide the output : show tech-support
Upload the tech-support report here under your service tag https://dell.to/3VvGNHA
Then Private Message me the service tag for me to retrieve the report.
You can click on my name : MOD Dell -Charles R
Look in the Right hand column:
Contact Me
Send this user a private message with service tag
DELL-Charles R
Moderator
Moderator
•
3.4K Posts
0
October 12th, 2022 05:00
Hello GregT_at_FCSG,
I received the private message. Thank you for the attachments. I'll get them and review and update you.
DELL-Charles R
Moderator
Moderator
•
3.4K Posts
0
October 12th, 2022 13:00
Hello GregT_at_FCSG,
From the core we can see the spanning tree TCN's on 1/0/7 and 1/0/48 connecting to switch RM113 and Switch RM 223.
Can you check to see what is happening on those two RM switches. Make sure that only the Core switch is connected to the problem switch RM119A via 1/48.
Also check if any interfaces are failing on those switches.
Can you give some history information:
Is this a configuration that used to work and now does not or is it a new config on the network?
If it is an existing configuration; Do you know of any network changes that were made around the time the issue started?
GregT_at_FCSG
5 Posts
0
October 16th, 2022 08:00
History:
Three switch domains/networks each terminating to their respective phsyicaly firewall port. The three domains are private, public, and voice. Multiple wiring closets with each closet have a switch in each domain and an uplink to their respective top level switch.
There was not a connection between any of the domains. All switches had an out of box (default) configuration with only a host name, IP address, gateway, username and password added.
All switches had single link between upstream and downstream switches. The deepest level is three switches. Most switches are 6248 but overtime, a few other Cisco Small Business (SG series) have been added as well as a few off brands.
Some users have 4-port unmanaged switches at the desktop.
Network was not monitored, logs were not reviewed. Any network problems may have been passed off as an anomoly or self resolved, thus unnoticed. I have to assume this network has existed for 10+ years.
June 2022, I became involved. I was asked to allow the PCs to attach to the VoIP phone data port. This would mean create VLANs and combining the data network and voice network into a single domain. These networks had different IP addresses.
The simple solution was to keep the data on VLAN1 and create a new VLAN (110) for the voice network. Then, start moving the phones from the dedicated voice switch to the data switch.
The voice VLAN would still use the firewall as the gateway so a VLAN 110 interface was not needed on the data switch. In fact, none of the switches were running layer 3, the firewall was the default gateway for every network.
To join the data and the voice network, I configured a port on the data switch as an access port with this configuration:
CoreSwitch#show running-config interface gi1/0/40
switchport access vlan 110
lldp transmit-tlv port-desc sys-name sys-desc sys-cap
lldp transmit-mgmt
lldp notification
CoreSwitch#show isdp neighbors det
Device ID PHSWITCH1
Address(es):
IP Address: xx.xx.110.3
Capability Router
Platform PCT6248P
Interface Gi1/0/40
Port ID 1/0/40
Holdtime 156
Advertisement Version 2
Time when last changed 0 days 00:01:04
Version :
3.2.0.7
The configuration of PHSWITCH1 is unknow at this time but assumed to be default.
This work for a month without apparent issue but since this a school, there was not much activity.
In August, the network had one switch drop off line im,mediately after trunking was enabled to allow the voice vlan to propagate. Problem woukd resolve when a docking station/iMAC attached was removed from the user port. Symptom was the downstream switch became ROOT. Moving the dock station/iMAC to an access port on the core switch resolved the issue.
In September, the same scenario happened on another switch. The symptom was the switch would stop responing to ping and traffic would not flow. Found disconnecting a specific user (port 44) on the downstream switch resolved the issue. Once resolved, the dock could be re-attached. Unlike the August failure, reattaching the dock to port 44 did not cause the problem to return. In the Augsust failure, the problem was reproducable as soon as the dock station was attached to the access switch (model 6248) but did cuase an issue when attached to the CoreSwitch (model 2048).
At this time, syslogging was enabled, spanning tree settings applied, network monitoring (ping tool and SNMP) added. Now that more information was available, I found that multiple switches would go offline and then reappear.
Spanning tree topology changes were frequent. More tuning with spanning tree happened, tyical settings now are:
global
spanning-tree portfast bpdufilter default
access ports:
spanning-tree portfast
spanning-tree tcnguard
Not all switches have been touched yet but this seems to be calming down the log messages
"Spanning Tree Topology Change: 0, Unit: 1"
and eliminated any:
"Spanning Tree Topology Change Received: MSTID: 0 Gi1/0/xx"
The Cisco SG switches are configured with
SWITCH-37#sho spanning-tree
Spanning tree enabled mode RSTP
Default port cost method: long
I am investigating (without much luck), what the setting is for the Dell 2048 and 6248 switches for the port cost method.
DELL-Joey C
Moderator
Moderator
•
3.2K Posts
0
October 16th, 2022 20:00
Hi @GregT_at_FCSG,
Based on your switch logs, I found that the switch is on 3.3.5.5. There is a fix in the firmware on 3.3.18.1 about voipTask that might be related to your situation. Try updating the firmware and check if the issue persist.
https://dell.to/3g2s0ni
DELL-Charles R
Moderator
Moderator
•
3.4K Posts
0
October 17th, 2022 13:00
Hello GregT_at_FCSG,
This may be a good resource for you:
Route Preferences Pages 95-98
https://dell.to/3SedUwp
PowerConnect 6248P Release Notes:
https://dell.to/3Tquz0W
GregT_at_FCSG
5 Posts
0
October 17th, 2022 13:00
We are putting a freeze any changes for a few weeks so I will not be able to try anything until mid- November.
I do need to understand the spanning-tree port/path cost method for the 6248 (short or long). We have both a mixed model and a mix vendor environment and i want to make sure they all match.
I can find this info on various Dell switches but have been unable to locate this for the PCT6248 and also did it change with any firmware revisions. Do you know where I can get this information?
GregT_at_FCSG
5 Posts
0
October 17th, 2022 14:00
Not sure why I didn't think of this earlier.
The newer version (a.k.a. long) will calcualte the cost of a 1G link to be 20,000 versus the older version (a.k.a. short) would calculate the cost to be 4. So all I had to do is look at the cost of a port and sure enough a 1G link is 20,000.
To answer my own question, the PCT6248 is using the long method.
GregT_at_FCSG
5 Posts
0
November 29th, 2022 13:00
I have not made any firmware updates or changes. Acess switch SWITCH_Rm113 (PCT6248P) downlinked from my CoreSwitch (N2048) decided to become STP Root for no reason. This caused the CoreSwitch to set the STP state to discard on the downlink port (Port 7) which is connected to SWITCH_Rm113 port 48.
The CoreSwitch STP priority is 4096. MAC address: F8B1.5654.8321
The SWITCH_Rm113 STP priority is 32768. MAC address: D067.E5AE.8712. Firmware 3.3.1.10
Why would a switch decide to become STP root for no apparent reason? No strange log entries around this same time.
Log from SWITCH_Rm113:
<189> NOV 29 13:37:26 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2831 %% Link Down: 1/0/6
<189> NOV 29 13:37:26 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2832 %% Link on 1/0/6 is failed
<189> NOV 29 13:37:26 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2833 %% 1/0/6 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 14:49:46 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2834 %% Link Down: 1/0/23
<189> NOV 29 14:49:46 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2835 %% Link on 1/0/23 is failed
<189> NOV 29 14:49:46 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2836 %% 1/0/23 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 14:52:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2837 %% Unit 1 elected as the new STP root
<189> NOV 29 14:52:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2838 %% Instance 0 has elected a new STP root: 8000:d067:e5ae:8712
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2839 %% Link Down: 1/0/48
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2840 %% Link on 1/0/48 is failed
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2841 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:02:03 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2842 %% Link Up: 1/0/48
<189> NOV 29 15:02:03 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2843 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:02:25 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2844 %% 1/0/48 is transitioned from the Learning state to the Forwarding state in instance 0
<189> NOV 29 15:02:25 192.168.0.34-1 TRAPMGR[125666288]: traputil.c(611) 2845 %% Spanning Tree Topology Change: 0, Unit: 1
<189> NOV 29 15:02:49 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2846 %% Link Down: 1/0/48
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2839 %% Link Down: 1/0/48
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2840 %% Link on 1/0/48 is failed
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2841 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:02:03 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2842 %% Link Up: 1/0/48
:
:
<189> NOV 29 15:06:23 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2855 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:06:32 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2856 %% Link Up: 1/0/48
<189> NOV 29 15:06:32 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2857 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:06:39 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2858 %% Instance 0 has elected a new STP root: 1000:f8b1:5654:8321
<189> NOV 29 15:06:39 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2859 %% 1/0/48 is transitioned from the Learning state to the Forwarding state in instance 0
<189> NOV 29 15:06:39 192.168.0.34-1 TRAPMGR[125666288]: traputil.c(611) 2860 %% Spanning Tree Topology Change: 0, Unit: 1
I shut/no shut CoreSwitch port 7 multiple times. After 13 min and 13 seconds, SWITCH_Rm113 changed STP root to the CoreSwith. Not sure if the multiple shut/no shut assisted with this change.
SWITCH_Rm113#show spanning-tree
Spanning tree :Enabled - BPDU Flooding :Disabled - Portfast BPDU filtering :Enabled - mode :rstp
CST Regional Root: 80:00:D0:67:E5:AE:87:12
Regional Root Path Cost: 0
ROOT ID
Address F8:B1:56:54:83:21
Path Cost 20000
Root Port 1/g48
Hello Time 2 Sec Max Age 20 sec Forward Delay 15 sec TxHoldCount 6 sec
Bridge ID
Priority 32768
Address D0:67:E5:AE:87:12
CoreSwitch#show spanning-tree
Spanning tree :Enabled - BPDU Flooding :Disabled - Portfast BPDU filtering :Disabled - mode :rstp
CST Regional Root: 10:00:F8:B1:56:54:83:21
Regional Root Path Cost: 0
ROOT ID
Priority 4096
Address F8B1.5654.8321
This Switch is the Root.
Hello Time 2 Sec Max Age 20 sec Forward Delay 15 sec TxHoldCount 6 sec
Bridge Max Hops 20
DELL-Young E
Moderator
Moderator
•
3.7K Posts
0
November 29th, 2022 18:00
Hello thanks for choosing Dell.
I suggest bringing firmware to up to date on both switches and check the status. It could be local loop occurred in the setup.