This post is more than 5 years old
3 Posts
0
13874
May 30th, 2017 06:00
Dell/Force10 S4048-ON LACP Port Flapping Before Reload
I have a pair of S4048 switches connected together using VLT which each have 2x uplinks to my core S6010 pair (also VLT). The servers connected to the 4048 pair do so with a link to each switch using LACP.
The problem I'm having is that when I reload either one of the 4048 switches it spends 120 seconds rapidly flapping all of the LACP links before actually rebooting. During this time I can also see syslog messages from my core 6010 switches saying they are throttling management traffic due to excessive control messages being received from the 4048 that I've told to reload.
The 4048s are running FTOS 9.10 (0.2) and the 6010s are running FTOS 9.10 (0.1)
A sample of the syslog messages from the 4048s are:
May 30 10:58:11 %STKUNIT1-M:CP %LACP-5-PORT-UNGROUPED: PortChannel-009-Ungrouped: Interface Te 1/9 exited port-channel 9.
May 30 10:58:11 %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Po 9
May 30 10:58:11 %STKUNIT1-M:CP %LACP-5-PORT-UNGROUPED: PortChannel-100-Ungrouped: Interface Fo 1/52 exited port-channel 100.
May 30 10:58:11 %STKUNIT1-M:CP %LACP-5-PORT-GROUPED: PortChannel-100-Grouped: Interface Fo 1/52 joined port-channel 100.
May 30 10:58:12 %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Po 100
May 30 10:58:12 %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Po 100
May 30 10:58:12 %STKUNIT1-M:CP %LACP-5-PORT-GROUPED: PortChannel-100-Grouped: Interface Fo 1/52 joined port-channel 100.
May 30 10:58:13 %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Po 100
May 30 10:58:13 %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Po 100
May 30 10:58:13 %STKUNIT1-M:CP %LACP-5-PORT-UNGROUPED: PortChannel-100-Ungrouped: Interface Fo 1/52 exited port-channel 100.
May 30 10:58:15 %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Po 9
May 30 10:58:15 %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Po 9
May 30 10:58:15 %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Po 9
May 30 10:58:15 %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Po 100
May 30 10:58:15 %STKUNIT1-M:CP %LACP-5-PORT-UNGROUPED: PortChannel-100-Ungrouped: Interface Fo 1/52 exited port-channel 100.
May 30 10:58:15 %STKUNIT1-M:CP %LACP-5-PORT-GROUPED: PortChannel-100-Grouped: Interface Fo 1/52 joined port-channel 100.
May 30 10:58:15 %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Po 100
May 30 10:58:16 %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Po 100
May 30 10:58:16 %STKUNIT1-M:CP %LACP-5-PORT-UNGROUPED: PortChannel-100-Ungrouped: Interface Fo 1/52 exited port-channel 100.
May 30 10:58:16 %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Po 9
May 30 10:58:17 %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Po 9
The config from interface Te 1/9 is:
interface TenGigabitEthernet 1/9
no ip address
mtu 9216
port-channel-protocol LACP
port-channel 9 mode active
no shutdown
....and the config from the port channel is:
interface Port-channel 9
no description
no ip address
mtu 9216
portmode hybrid
switchport
spanning-tree rstp edge-port bpduguard shutdown-on-violation
lacp fast-switchover
vlt-peer-lag port-channel 9
no shutdown
I'd appreciate any insight anyone can give on this.
Thanks.
packet_wrangler
3 Posts
0
June 1st, 2017 08:00
I've finally fixed this. The problem was with both the rack switch and the core switch having their LACP config set as 'Active'. Setting one end (the rack switch ports) to 'Passive' was the solution here. Having both ends set to active was causing one end or the other (or both) to continually try to bring the port channel back up when the switch was trying to shut it down.
DELL-Josh Cr
Community Manager
•
9.7K Posts
•
43.4K Points
0
May 31st, 2017 13:00
Hi,
Is it just on the LACP ports that go to servers or is it also to ones that go to the 6010s? What happens if you set a static lag instead of lacp? Page 774 http://downloads.dell.com/Manuals/all-products/esuprt_ser_stor_net/esuprt_networking/esuprt_net_fxd_prt_swtchs/force10-s4048-on_Administrator%20Guide_en-us.pdf
packet_wrangler
3 Posts
0
June 1st, 2017 02:00
Hi Josh,
I'm using LACP on the port channels between the 4048 and the 6010 switches. The documentation I have read said that a static lag should only be used for the VLTi links and that LACP should be used for the links to other switches. Is this not the case?
To further test this problem I've set up a small test environment using 4x 4048 switches (I don't have any more 6010s). I've set 2 of these as the 'core' switches and 2 as the 'rack' switches using the same config, the same number of links and the same firmware as in the live environment. I've also connected a server to the 2 rack switches (using LACP). I reloaded one of the rack switches 10 times in a row (no config changes in between) and got it to re-produce the port-flapping problem twice. The other 8 times it reloaded without any port flapping.
The 4048 in my live environment now has nothing connected to it other than a single uplink to the core and has been updated to firmware 9.11(2.0P0). Reloading it now causes it to flap the physical port and the port channel it belongs to for only 15 seconds (rather than 2 minutes), but it it reports "All VLT LAG's gracefully shut down...!!!" which it didn't before.
Thanks for your help
Stuart
DELL-Josh Cr
Community Manager
•
9.7K Posts
•
43.4K Points
0
June 1st, 2017 10:00
I am glad that you were able to get it resolved.
ronald03
1 Rookie
•
1 Message
•
2 Points
0
May 29th, 2024 13:51
I have a problem on a dell S4048-N switch with reference to these logs:
May 26 20:27:20 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Po 20
May 26 20:22:18 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Te 1/20
May 26 20:22:17 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Te 1/20
May 26 20:22:07 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Te 1/20
May 26 20:22:05 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Te 1/20
May 26 20:21:58 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Te 1/20
May 26 20:21:56 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Te 1/20
May 26 20:21:25 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Po 20
May 26 20:21:25 UTC-4: %STKUNIT1-M:CP %LACP-5-PORT-UNGROUPED: PortChannel-020-Ungrouped: Interface Te 1/20 exited port-channel 20.
May 12 09:45:49 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Po 20
May 12 09:45:49 UTC-4: %STKUNIT1-M:CP %LACP-5-PORT-GROUPED: PortChannel-020-Grouped: Interface Te 1/20 joined port-channel 20.
May 12 05:55:14 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Te 1/20
May 12 05:55:13 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Te 1/20
May 12 05:50:02 UTC-4: %STKUNIT1-M:CP %LACP-5-PORT-UNGROUPED: PortChannel-020-Ungrouped: Interface Te 1/20 exited port-channel 20.
May 12 05:50:02 UTC-4: %STKUNIT1-M:CP %IFMGR-5-OSTATE_DN: Changed interface state to down: Po 20
This port 20 is associated with port-channel 20 and it has a connection to a DELL server. In the interface I have the configuration:
port-channel-protocol LACP
port-channel 20 mode active