Networking General

Last reply by 11-29-2022 Unsolved
Start a Discussion
2 Bronze
2 Bronze
348

PowerConnect 6248 PCT6248 stops forwarding traffic intermittently

Management on VLAN 1 and user traffic on VLAN 1 stops.  Although I have VLANs configured, I am only using VLAN 1.

The log on SWITCH_119A shows nothing.  I only have three user connections on this switch, ports 13, 16, & 44.  Through trial and error, I have that if I bounce port 44 (SHUT/NO SHUT), the switch immediately begins transmitting traffic.  The failure is sporadic, a few times per week.  Sometimes during the day, other times in the early morning hours when no users are in.

I have an identical switch SWITCH_119B in the same rack.  I added SWITCH_119B to the same rack and with its own uplink to the core for troubleshooting.  When SWITCH_119A would fail, I would move my patch cables from switch A to B on the corresponding port.  When doing this, I discovered that when port 44 was disconnected, "A" would heal.

When I left my users on "B", it experienced the same failure. Disconnecting (or SHUT/NO SUT) port 44 would allow it to heal.

I now have all but 3 connections on switch "B" and it is not failing.

Port 44 has a docking station that allows various Dell laptops to connect.

Switch "A" has the remaining 3 connections.

When switch "A" fails, since it is isolated, it comes the spanning-tree root for itself.  This caused the CoreSwitch to disable the downlink port 47 to switch "A" as a spanning-tree discard state.  Thus, it appears BPDU is passing but not production traffic.  I disabled spanning-tree on the CoreSwitch downlink port 47 to switch "A" to eliminate a possible spanning-tree issue.

~~~~~~~~~~~~~~~~~~~~~~~

SWITCH_119A#show system

System Description: Dell Ethernet Switch

System Up Time: 7 days, 19h:02m:08s

System Contact:

System Name: SWITCH_119A

System Location:

Burned In MAC Address: D067.E5A3.6BDF

System Object ID: 1.3.6.1.4.1.674.10895.3013

System Model ID: PCT6248P

Machine Type: PowerConnect 6248P

 

 

SWITCH_119A#show running-config interface ethernet 1/g47

switchport mode general

switchport general allowed vlan 63,110,200 tagged

lldp transmit-tlv port-desc sys-name sys-desc sys-cap

lldp transmit-mgt

 

SWITCH_119A#show isdp neighbors

 

Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge,

                  S - Switch, H - Host, I - IGMP, r - Repeater

Device ID              Intf    Holdtime  Capability   Platform             Port ID              

---------------------- ------- --------- ------------ -------------------- ----------------------

CoreSwitch             1/g48   145       R            N2048                Gi1/0/47             

 

~~~~~~~~~~~~~~~~~~~~~~~

 

CoreSwitch#show system

 

System Description: Dell Networking Switch

System Up Time: 55 days, 22h:54m:06s

System Contact: Network_Admin

System Name: CoreSwitch

System Location: Room 223

Burned In MAC Address: F8B1.5654.8321

System Object ID: 1.3.6.1.4.1.674.10895.3054

System Model ID: N2048

Machine Type: Dell Networking N2048

 

CoreSwitch#show isdp neigh

 

Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge,

                  S - Switch, H - Host, I - IGMP, r - Repeater

Device ID                Intf      Holdtime  Capability Platform         Port ID

------------------------ --------- --------- ---------- ---------------- ---------

SWITCH_119B              Gi1/0/4   155       R          PCT6248P         1/0/48

SWITCH_Rm113             Gi1/0/7   179       R          PCT6248P         1/0/48

SWITCH_Rm207             Gi1/0/21  156       R          PCT6248P         1/0/48

PHSWITCH1                Gi1/0/40  165       R          PCT6248P         1/0/40

Switch98B9D1             Gi1/0/42  143       S I        Cisco SG220-28MP gi25

Switch98BBA1             Gi1/0/42  155       S I        Cisco SG220-28MP gi25

Switch98BF41             Gi1/0/42  118       S I        Cisco SG220-28MP gi25

SWITCH2_Rm223            Gi1/0/46  157       R          PCT6248P         1/0/44

SWITCH_119A              Gi1/0/47  154       R          PCT6248P         1/0/48

SWITCH3_Rm223            Gi1/0/48  160       R          PCT6248P         1/0/48

SWITCH_WestBldg          Te1/0/1   160       R          PCT6248P         1/0/45

 

CoreSwitch#show running-config interface gi1/0/47

description "SWITCH119A"

spanning-tree disable

switchport mode general

switchport general allowed vlan 63,110,200 tagged

 

 

 

Replies (10)
318

Hello GregT_at_FCSG,

 

I would like to gather some more information.

 

Can you provide a topology layout?

 

Also run this and provide the output : show tech-support

 

Upload the  tech-support  report here under your service tag  https://dell.to/3VvGNHA

 

Then Private Message me the service tag for me to retrieve the report.

You can click on my name : MOD Dell -Charles R

Look in the Right hand column:

     Contact Me

          Send this user a private message with service tag

 

 


Dell -Charles R
Social Media and Communities Professional
Dell Technologies | Enterprise Support Services
#IWork4Dell

Did I answer your query? Please click on ‘Accept as Solution’. ‘Kudo’ the posts you like!

305

Hello GregT_at_FCSG,

 

I received the private message. Thank you for the attachments. I'll get them and review and update you.


Dell -Charles R
Social Media and Communities Professional
Dell Technologies | Enterprise Support Services
#IWork4Dell

Did I answer your query? Please click on ‘Accept as Solution’. ‘Kudo’ the posts you like!

302

Hello GregT_at_FCSG,

 

From the core we can see the spanning tree TCN's on 1/0/7 and 1/0/48 connecting to switch RM113 and Switch RM 223.

 

Can you check to see what is happening on those two RM switches. Make sure that only the Core switch is connected to the problem switch RM119A via 1/48.

 

Also check if any interfaces are failing on those switches.

 

 

Can you give some history information:

 

Is this a configuration that used to work and now does not or is it a new config on the network?

 

If it is an existing configuration; Do you know of any network changes that were made around the time the issue started?


Dell -Charles R
Social Media and Communities Professional
Dell Technologies | Enterprise Support Services
#IWork4Dell

Did I answer your query? Please click on ‘Accept as Solution’. ‘Kudo’ the posts you like!

2 Bronze
2 Bronze
261

History:

Three switch domains/networks each terminating to their respective phsyicaly firewall port.  The three domains are private, public, and voice.  Multiple wiring closets with each closet have a switch in each domain and an uplink to their respective top level switch.

There was not a connection between any of the domains.  All switches had an out of box (default)  configuration with only a host name, IP address, gateway, username and password added.

All switches had single link between upstream and downstream switches.  The deepest level is three switches. Most switches are 6248 but overtime, a few other Cisco Small Business (SG series) have been added as well as a few off brands.

Some users have 4-port unmanaged switches at the desktop.

Network was not monitored, logs were not reviewed.  Any network problems may have been passed off as an anomoly or self resolved, thus unnoticed.  I have to assume this network has existed for 10+ years.

June 2022, I became involved.  I was asked to allow the PCs to attach to the VoIP phone data port.  This would mean create VLANs and combining the data network and voice network into a single domain.  These networks had different IP addresses.

The simple solution was to keep the data on VLAN1 and create a new VLAN (110) for the voice network.  Then, start moving the phones from the dedicated voice switch to the data switch.

The voice VLAN would still use the firewall as the gateway so a VLAN 110 interface was not needed on the data switch.  In fact, none of the switches were running layer 3, the firewall was the default gateway for every network.

To join the data and the voice network, I configured a port on the data switch as an access port with this configuration:

CoreSwitch#show running-config interface gi1/0/40

switchport access vlan 110
lldp transmit-tlv port-desc sys-name sys-desc sys-cap
lldp transmit-mgmt
lldp notification

CoreSwitch#show isdp neighbors det

<snip>

Device ID PHSWITCH1
Address(es):
IP Address: xx.xx.110.3
Capability Router
Platform PCT6248P
Interface Gi1/0/40
Port ID 1/0/40
Holdtime 156
Advertisement Version 2
Time when last changed 0 days 00:01:04
Version :
3.2.0.7

<snip>

The configuration of PHSWITCH1 is unknow at this time but assumed to be default.

This work for a month without apparent issue but since this a school, there was not much activity.

In August, the network had one switch drop off line im,mediately after trunking was enabled to allow the voice vlan to propagate.  Problem woukd resolve when a docking station/iMAC attached was removed from the user port.  Symptom was the downstream switch became ROOT.  Moving the dock station/iMAC to an access port on the core switch resolved the issue.

In September, the same scenario happened on another switch.  The symptom was the switch would stop responing to ping and traffic would not flow.  Found disconnecting a specific user (port 44) on the downstream switch resolved the issue.  Once resolved, the dock could be re-attached.  Unlike the August failure, reattaching the dock to port 44 did not cause the problem to return.  In the Augsust failure, the problem was reproducable as soon as the dock station was attached to the access switch (model 6248) but did cuase an issue when attached to the CoreSwitch (model 2048).

At this time, syslogging was enabled, spanning tree settings applied, network monitoring (ping tool and SNMP) added.  Now that more information was available, I found that multiple switches would go offline and then reappear.

Spanning tree topology changes were frequent.  More tuning with spanning tree happened, tyical settings now are:

global

spanning-tree portfast bpdufilter default

access ports:

spanning-tree portfast
spanning-tree tcnguard

Not all switches have been touched yet but this seems to be calming down the  log messages

"Spanning Tree Topology Change: 0, Unit: 1"

and eliminated any:

"Spanning Tree Topology Change Received: MSTID: 0 Gi1/0/xx"

The Cisco SG switches are configured with

SWITCH-37#sho spanning-tree

Spanning tree enabled mode RSTP
Default port cost method: long

I am investigating (without much luck), what the setting is for the Dell 2048 and 6248 switches for the port cost method.

 

256

Hi @GregT_at_FCSG,

 

Based on your switch logs, I found that the switch is on 3.3.5.5. There is a fix in the firmware on 3.3.18.1 about voipTask that might be related to your situation. Try updating the firmware and check if the issue persist. 

 

https://dell.to/3g2s0ni


DELL-Joey C
Social Media and Communities Professional
Dell Technologies | Enterprise Support Services
#IWork4Dell

Did I answer your query? Please click on ‘Accept as Solution’. ‘Kudo’ the posts you like!

247

We are putting a freeze any changes for a few weeks so I will not be able to try anything until mid- November.

I do need to understand the spanning-tree port/path cost method for the 6248 (short or long).  We have both a mixed model and a mix vendor environment and i want to make sure they all match.

I can find this info on various Dell switches but have been unable to locate this for the PCT6248 and also did it change with any firmware revisions.  Do you know where I can get this information?

247

Hello GregT_at_FCSG,

 

This may be a good resource for you:

Route Preferences  Pages 95-98

https://dell.to/3SedUwp

 

PowerConnect 6248P Release Notes:

https://dell.to/3Tquz0W

 


Dell -Charles R
Social Media and Communities Professional
Dell Technologies | Enterprise Support Services
#IWork4Dell

Did I answer your query? Please click on ‘Accept as Solution’. ‘Kudo’ the posts you like!

246

Not sure why I didn't think of this earlier.

The newer version (a.k.a. long) will calcualte the cost of a 1G link to be 20,000 versus the older version (a.k.a. short) would calculate the cost to be 4.  So all I had to do is look at the cost of a port and sure enough a 1G link is 20,000.

To answer my own question, the PCT6248 is using the long method.

 

2 Bronze
2 Bronze
167

I have not made any firmware updates or changes.  Acess switch SWITCH_Rm113 (PCT6248P) downlinked from my CoreSwitch (N2048) decided to become STP Root for no reason.  This caused the CoreSwitch to set the STP state to discard on the downlink port (Port 7) which is connected to SWITCH_Rm113 port 48.

 

The CoreSwitch STP priority is 4096.  MAC address: F8B1.5654.8321

The SWITCH_Rm113 STP priority is 32768. MAC address: D067.E5AE.8712. Firmware 3.3.1.10

Why would a switch decide to become STP root for no apparent reason?  No strange log entries around this same time.

 

Log from SWITCH_Rm113:

<189> NOV 29 13:37:26 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2831 %% Link Down: 1/0/6
<189> NOV 29 13:37:26 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2832 %% Link on 1/0/6 is failed
<189> NOV 29 13:37:26 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2833 %% 1/0/6 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 14:49:46 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2834 %% Link Down: 1/0/23
<189> NOV 29 14:49:46 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2835 %% Link on 1/0/23 is failed
<189> NOV 29 14:49:46 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2836 %% 1/0/23 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 14:52:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2837 %% Unit 1 elected as the new STP root
<189> NOV 29 14:52:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2838 %% Instance 0 has elected a new STP root: 8000:d067:e5ae:8712

<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2839 %% Link Down: 1/0/48
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2840 %% Link on 1/0/48 is failed
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2841 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:02:03 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2842 %% Link Up: 1/0/48
<189> NOV 29 15:02:03 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2843 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:02:25 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2844 %% 1/0/48 is transitioned from the Learning state to the Forwarding state in instance 0
<189> NOV 29 15:02:25 192.168.0.34-1 TRAPMGR[125666288]: traputil.c(611) 2845 %% Spanning Tree Topology Change: 0, Unit: 1
<189> NOV 29 15:02:49 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2846 %% Link Down: 1/0/48


<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2839 %% Link Down: 1/0/48
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2840 %% Link on 1/0/48 is failed
<189> NOV 29 15:01:58 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2841 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:02:03 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2842 %% Link Up: 1/0/48

    :

    :

<189> NOV 29 15:06:23 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2855 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:06:32 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2856 %% Link Up: 1/0/48
<189> NOV 29 15:06:32 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2857 %% 1/0/48 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> NOV 29 15:06:39 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2858 %% Instance 0 has elected a new STP root: 1000:f8b1:5654:8321
<189> NOV 29 15:06:39 192.168.0.34-1 TRAPMGR[152677424]: traputil.c(611) 2859 %% 1/0/48 is transitioned from the Learning state to the Forwarding state in instance 0
<189> NOV 29 15:06:39 192.168.0.34-1 TRAPMGR[125666288]: traputil.c(611) 2860 %% Spanning Tree Topology Change: 0, Unit: 1

I shut/no shut CoreSwitch port 7 multiple times. After 13 min and 13 seconds, SWITCH_Rm113 changed STP root to the CoreSwith. Not sure if the multiple shut/no shut assisted with this change.

SWITCH_Rm113#show spanning-tree
Spanning tree :Enabled - BPDU Flooding :Disabled - Portfast BPDU filtering :Enabled - mode :rstp
CST Regional Root: 80:00:D0:67:E5:AE:87:12
Regional Root Path Cost: 0
ROOT ID
Address F8:B1:56:54:83:21
Path Cost 20000
Root Port 1/g48
Hello Time 2 Sec Max Age 20 sec Forward Delay 15 sec TxHoldCount 6 sec
Bridge ID
Priority 32768
Address D0:67:E5:AE:87:12

CoreSwitch#show spanning-tree

Spanning tree :Enabled - BPDU Flooding :Disabled - Portfast BPDU filtering :Disabled - mode :rstp
CST Regional Root: 10:00:F8:B1:56:54:83:21
Regional Root Path Cost: 0
ROOT ID
Priority 4096
Address F8B1.5654.8321
This Switch is the Root.
Hello Time 2 Sec Max Age 20 sec Forward Delay 15 sec TxHoldCount 6 sec
Bridge Max Hops 20

Latest Solutions
Top Contributor