pzero
2 Iron

PowerConnect 6248 LAG and spanning tree protocol

Hi,

We have two PowerConnect 6248 switches interconnected with a LAG of two 10GbE connections.

Should we enable the spanning tree protocol on the LAG? Pros and cons?

Should we also enable it on the ports underlying the LAG?

Thanks.

0 Kudos
19 Replies
bh1633
3 Cadmium

Re: PowerConnect 6248 LAG and spanning tree protocol

A LAG is just like a port.  If STP is enabled on the switch, then it is enabled on the LAG by default.

For PowerConnect switches, when a port is in a LAG, its port specific configuration is not used.  The configuration of the LAG is applied to the member ports.

Whether or not to enable STP on any port or switch depends on the requirements of your network.  PROS: no loops, redundancy.  CONS: possible loops

0 Kudos
pzero
2 Iron

Re: PowerConnect 6248 LAG and spanning tree protocol

If the two ports LAG is the only interconnection between the two switches, is it safe to not enable STP?

We're going to use the two PowerConnect switches for connecting ESX servers and EqualLogic arrays: each server/array will have at least one connection to each switch for redundancy.

Would you still enable STP in such scenario?

What about the portfast option?

Thanks.

0 Kudos
pzero
2 Iron

Re: PowerConnect 6248 LAG and spanning tree protocol

What would be the benefits of using LACP in such setup and would you recommend it?

Thanks.

0 Kudos
bh1633
3 Cadmium

Re: PowerConnect 6248 LAG and spanning tree protocol

If there is no chance of a loop, you can disable STP.

Connecting multiple NICS from the servers and arrays to each of the switches will not create a loop since the NICS will not bridge traffic.

LACPs main benefit is to help prevent cabling mistakes.  It adds complexity to your simple set up, so I would not bother.

0 Kudos
pzero
2 Iron

Re: PowerConnect 6248 LAG and spanning tree protocol

I have STP enabled as by default on the two PowerConnect 6248 switches interconnected with the LAG (2x1GbE) and I hit an issue while testing the redundancy of the network.

After rebooting root STP switch, when it comes back online, all the ports on the other switch will stop working for a few seconds (e.g. pinging the other switch management IP address from a server directly connected to it won't get any replies for a few seconds, then it will get back working as normal).

Since they are two separate switches connected with a LAG, I didn't expect that rebooting one of them could also affect the other!

(BTW, if I reboot the other switch, the ports on switch with is root for STP won't suffer any interruption)

 

I'm including the console logs from the switches in question below: switch2 is the root STP switch that I reload and switch1 is the other switch which suffers the issue when the first switch comes back online, LAG 3 is the interconnection.

 

Is this behavior normal? Any way to prevent the issue (besides completely disabling STP)?

What I'm looking is a way to completely manage the two switches separately for a fully redundant network (of course each server and SAN array will have a connection to one switch and another connection to the other switch) and rebooting one shouldn't bring down connectivity on the ports of the other.

Thanks.

 

switch2#
reload
*** ... CUT CUT ... ***
(Unit 1 - Waiting to select management unit)>
Applying configuration, please wait ...................................................................................
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 33 % Unit 1 Port 45 is removed from the trunk LAG- 2
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 34 % Unit 1 Port 46 is removed from the trunk LAG- 2
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 35 % Unit 1 Port 47 is removed from the trunk LAG- 1
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 36 % Unit 1 Port 48 is removed from the trunk LAG- 1
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 38 % Unit 1 Port 49 is removed from the trunk LAG- 3
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 40 % Unit 1 Port 50 is removed from the trunk LAG- 3
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 66 % Link on LAG- 2 is failed
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 67 % Link on LAG- 2 is failed
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 68 % Link on LAG- 1 is failed
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 69 % Link on LAG- 1 is failed
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 70 % Link on LAG- 3 is failed
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 71 % Link on LAG- 3 is failed
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 72 % Unit 1 Port 49 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[250907120]: traputil.c(906) 73 % Link Up: Unit 1 Port 49
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 74 % Unit 1 Port 50 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> JAN 01 00:00:28 192.168.10.35-1 TRAPMGR[250907120]: traputil.c(906) 75 % Link Up: Unit 1 Port 50
<189> JAN 01 00:00:29 192.168.10.35-1 TRAPMGR[231121584]: traputil.c(906) 76 % Spanning Tree Topology Change: 0, Unit: 1
<189> JAN 01 00:00:29 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 77 % Unit 1 Port 49 is transitioned from the Learning state to the Forwarding state in instance 0
<189> JAN 01 00:00:31 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 78 % Unit 1 Port 11 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> JAN 01 00:00:31 192.168.10.35-1 TRAPMGR[250907120]: traputil.c(906) 79 % Link Up: Unit 1 Port 11
<189> JAN 01 00:00:31 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 80 % Unit 1 Port 49 is removed from the trunk LAG- 3
<189> JAN 01 00:00:31 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 81 % Unit 1 Port 50 is removed from the trunk LAG- 3
<189> JAN 01 00:00:31 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 82 % Unit 1 Port 49 is added to the trunk LAG- 3
<189> JAN 01 00:00:31 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 83 % Unit 1 Port 50 is added to the trunk LAG- 3
<189> JAN 01 00:00:32 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 88 % Link on LAG- 3 is failed
<189> JAN 01 00:00:32 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 89 % Link on LAG- 3 is failed
<189> JAN 01 00:00:32 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 90 % Unit 1 Port 49 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> JAN 01 00:00:32 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 91 % LAG- 3 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> JAN 01 00:00:32 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 92 % Unit 1 Port 50 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> JAN 01 00:00:32 192.168.10.35-1 TRAPMGR[231121584]: traputil.c(906) 93 % Spanning Tree Topology Change: 0, Unit: 1
<189> JAN 01 00:00:32 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 94 % LAG- 3 is transitioned from the Learning state to the Forwarding state in instance 0
<189> JAN 01 00:00:33 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 95 % Unit 1 Port 2 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> JAN 01 00:00:33 192.168.10.35-1 TRAPMGR[250907120]: traputil.c(906) 96 % Link Up: Unit 1 Port 2
<189> JAN 01 00:00:34 192.168.10.35-1 TRAPMGR[235924000]: traputil.c(906) 97 % Cold Start: Unit: 0
<189> JAN 01 00:01:01 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 98 % Unit 1 Port 11 is transitioned from the Learning state to the Forwarding state in instance 0
<189> JAN 01 00:01:03 192.168.10.35-1 TRAPMGR[214618864]: traputil.c(906) 99 % Unit 1 Port 2 is transitioned from the Learning state to the Forwarding state in instance 0
<189> JAN 01 00:01:14 192.168.10.35-1 TRAPMGR[263779952]: traputil.c(906) 100 % Unit 1 is the new stack master, Old stack master unit is 0

 

 

 

.
.
.
.
.
switch1#
<189> MAY 26 21:59:02 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 302 % Unit 1 Port 49 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:02 192.168.10.34-1 TRAPMGR[250907072]: traputil.c(906) 303 % Link Up: Unit 1 Port 49
<189> MAY 26 21:59:02 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 304 % Unit 1 Port 50 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:02 192.168.10.34-1 TRAPMGR[250907072]: traputil.c(906) 305 % Link Up: Unit 1 Port 50
<189> MAY 26 21:59:03 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 306 % Unit 1 Port 49 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:03 192.168.10.34-1 TRAPMGR[250907072]: traputil.c(906) 307 % Link Down: Unit 1 Port 49
<189> MAY 26 21:59:03 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 308 % Link on Unit 1 Port 49 is failed
<189> MAY 26 21:59:03 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 309 % Unit 1 Port 50 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:03 192.168.10.34-1 TRAPMGR[250907072]: traputil.c(906) 310 % Link Down: Unit 1 Port 50
<189> MAY 26 21:59:03 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 311 % Link on Unit 1 Port 50 is failed
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 314 % Unit 1 Port 49 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[250907072]: traputil.c(906) 315 % Link Up: Unit 1 Port 49
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 316 % Unit 1 Port 50 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[250907072]: traputil.c(906) 317 % Link Up: Unit 1 Port 50
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 318 % Instance 0 has elected a new STP root: 8000:YYYY:YYYY:YYYY
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 319 % Unit 1 Port 2 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 320 % Unit 1 Port 25 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 321 % LAG- 1 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 322 % LAG- 2 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[231121504]: traputil.c(906) 323 % Spanning Tree Topology Change: 0, Unit: 1
<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 324 % Unit 1 Port 49 is transitioned from the Learning state to the Forwarding state in instance 0
***PING REPLY STOPS AROUND HERE***
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 325 % Unit 1 Port 49 is removed from the trunk LAG- 3
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 326 % Unit 1 Port 50 is removed from the trunk LAG- 3
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 327 % Link on LAG- 3 is failed
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 328 % Unit 1 Port 49 is added to the trunk LAG- 3
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 329 % Link on LAG- 3 is failed
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 330 % Unit 1 Port 50 is added to the trunk LAG- 3
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 331 % Unit 1 Port 49 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 332 % Unit 1 Port 50 is transitioned from the Learning state to the Forwarding state in instance 0
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 333 % LAG- 3 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[231121504]: traputil.c(906) 334 % New Spanning Tree Root: 0, Unit: 1
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 335 % Unit 1 Port 50 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 336 % Unit 1 elected as the new STP root
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 337 % Instance 0 has elected a new STP root: 8000:XXXX:XXXX:XXXX
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 338 % Instance 0 has elected a new STP root: 8000:YYYY:YYYY:YYYY
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 339 % LAG- 1 is transitioned from the Learning state to the Forwarding state in instance 0
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 340 % LAG- 1 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:32 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 341 % LAG- 2 is transitioned from the Learning state to the Forwarding state in instance 0
<189> MAY 26 21:59:33 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 342 % LAG- 2 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> MAY 26 21:59:33 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 343 % LAG- 3 is transitioned from the Learning state to the Forwarding state in instance 0
***PING REPLY STARTS WORKING AGAIN AROUND HERE***
<189> MAY 26 21:59:59 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 345 % Unit 1 Port 2 is transitioned from the Learning state to the Forwarding state in instance 0
<189> MAY 26 21:59:59 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 346 % Unit 1 Port 25 is transitioned from the Learning state to the Forwarding state in instance 0
<189> MAY 26 22:00:02 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 347 % LAG- 1 is transitioned from the Learning state to the Forwarding state in instance 0
<189> MAY 26 22:00:08 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 348 % LAG- 2 is transitioned from the Learning state to the Forwarding state in instance 0

 

 

0 Kudos
pzero
2 Iron

Re: PowerConnect 6248 LAG and spanning tree protocol

I did some research, but I'm still not sure if what I'm seeing is the expected behavior or if it's a incorrect configuration ...

And besides completely disabling STP, I still didn't find a way to prevent the issue.

Any help would be appreciated.

Thanks.

0 Kudos
Highlighted
peterhd
1 Copper

Re: PowerConnect 6248 LAG and spanning tree protocol

Yes, this is fully expected behaviour. Look at these lines from your log:

 

<189> MAY 26 21:59:29 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 319 % Unit 1 Port 2 is transitioned from the Forwarding state to the Blocking state in instance 0

***PING REPLY STOPS AROUND HERE***

***PING REPLY STARTS WORKING AGAIN AROUND HERE***

<189> MAY 26 21:59:59 192.168.10.34-1 TRAPMGR[214618784]: traputil.c(906) 345 % Unit 1 Port 2 is transitioned from the Learning state to the Forwarding state in instance 0

 

Topology change puts by default all ports into blocking state and STP needs 30 seconds to move a port into forwarding state when no BPDUs are received.

For your configuration, there's no need for spanning tree. But if you still want to use it, observe these rules:

- use RSTP for fast convergence

- configure all end-station/server ports with "spanning-tree portfast" to ensure that topology changes don't affect them

- ports that interconnect switches must not be configured with "spanning-tree portfast"

0 Kudos
pzero
2 Iron

Re: PowerConnect 6248 LAG and spanning tree protocol

We'll also have to connect a Juniper firewall to each switch and the two Juniper firewalls will be interconnect between them with an ethernet cable for HA (using NSRP).

Is it still safe to completely disable spanning tree on the two switches for our setup?

Also, for some further testing, we connected an older PC5324 switche to the PC6248 switches using a LAG (2 x 1GbE ports)  and we didn't experience any network disruption on it while rebooting the STP root PC6248 switch (while we still experienced the issue on the other PC6248 switch). The PC5324 appear to be using the default spanning tree settings. Any ideas why it doesn't seem to suffer the same issue of the newer switch?

Thanks.

0 Kudos
pzero
2 Iron

Re: PowerConnect 6248 LAG and spanning tree protocol

After upgrading the two PowerConnect 6248 from firmware version 2.2.0.3 to 3.2.0.7, keeping the same config, it appears that there is an improvement with network downtime issue on the other switch when rebooting the STP root switch: the ping test from above loses about 7-8 replies, while it didn't get any reply for about 30 seconds with the previous firmware version.

We're still leaning towards completely disabling STP on the two switches.

Is that still safe even connecting the Juniper firewalls as outlined in my previous post?

Also, I still wonder why the older PowerConnect 5324 doesn't exhibit the same issue as the newer PowerConnect 6248 ...

Thanks.

0 Kudos