UNSOLVED
Network dropouts when disconnecting Stack UpLink Ports ( SDM template )
Hi guys
I've got 3 Powerconnect 8024f's in a stacked setup, using just a simple loop stack where one swtich plugs into the other using 1 port each.. (each swtich has 2 stacking ports, one going to each other switch).
The issue I have is, if I disconnect a stacking port I get never ending ping dropouts across devices connected to the switch. These devices are hypervisors running VMWare, each connected accross 2 of the switches. The dropouts stop once the stack port is plugged back in. Each stack port once disconnected makes different vmware machines dropout as such.
I've taken logs from when it happend and have tried rebuilding the switches ( upgrading to latest firmware, creating a new stack then copy pasting configuration back into the swithces ) and still the same issue.
Excuse the length,
Logging is enabled
Console Logging: Level warnings. Messages : 9 logged, 4562 ignored
Monitor Logging: disabled
Buffer Logging: Level informational. Messages : 314 logged, 4257 ignored
File Logging: Level emergencies. Messages : 0 logged, 4571 ignored
Switch Auditing : enabled
CLI Command Logging: disabled
Web Session Logging : disabled
SNMP Set Command Logging : disabled
Logging facility level : local7
Syslog Server Details:
10.0.5.102 : Level informational. Messages : 6 dropped
1650 Messages dropped due to lack of resources
Buffer Log:
<190> DEC 18 14:29:48 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 457 1 %% [CLI:root:10.0.0.193] User root logged in to enable mode.
<190> DEC 18 14:29:44 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 457 0 %% [CLI:root:10.0.0.193] User has succesfully logged in
<190> DEC 18 14:25:03 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 907 %% [CLI:root:10.0.0.193] User root logged in to enable mode.
<190> DEC 18 14:24:58 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 906 %% [CLI:root:10.0.0.193] User has succesfully logged in
<190> DEC 18 14:22:22 10.0.1.254-1 UNITMGR[228876048]: unitmgr.c(6506) 905 %% Co py of running configuration to backup unit complete
<189> DEC 18 14:22:06 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 904 %% Te 2/0/22 is transitioned from the Learning state to the Forwarding state in instan ce 0
<189> DEC 18 14:22:05 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 903 %% Po 4 is transitioned from the Learning state to the Forwarding state in instance 0
--More-- or (q)uit
OFF-STK-01#
OFF-STK-01#
OFF-STK-01#
OFF-STK-01#
OFF-STK-01#
OFF-STK-01#
OFF-STK-01#show logging
Logging is enabled
Console Logging: Level warnings. Messages : 9 logged, 4562 ignored
Monitor Logging: disabled
Buffer Logging: Level informational. Messages : 314 logged, 4257 ignored
File Logging: Level emergencies. Messages : 0 logged, 4571 ignored
Switch Auditing : enabled
CLI Command Logging: disabled
Web Session Logging : disabled
SNMP Set Command Logging : disabled
Logging facility level : local7
Syslog Server Details:
10.0.5.102 : Level informational. Messages : 6 dropped
1650 Messages dropped due to lack of resources
Buffer Log:
<190> DEC 18 14:29:48 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 4571 %% [CLI:root:10.0.0.193] User root logged in to enable mode.
<190> DEC 18 14:29:44 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 4570 %% [CLI:root:10.0.0.193] User has succesfully logged in
<190> DEC 18 14:25:03 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 907 %% [CLI:root:10.0.0.193] User root logged in to enable mode.
<190> DEC 18 14:24:58 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 906 %% [CLI:root:10.0.0.193] User has succesfully logged in
<190> DEC 18 14:22:22 10.0.1.254-1 UNITMGR[228876048]: unitmgr.c(6506) 905 %% Copy of running configuration to backup unit complete
<189> DEC 18 14:22:06 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 904 %% Te2/0/22 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:22:05 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 903 %% Po4 is transitioned from the Learning state to the Forwarding state in instance 0
--More-- or (q)uit
<189> DEC 18 14:22:05 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 902 %% Po4 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:22:05 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 901 %% Te2/0/24 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:22:05 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 900 %% Te2/0/24 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:22:05 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 899 %% Link Up: Te2/0/24
<189> DEC 18 14:22:03 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 898 %% Te2/0/22 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:22:03 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 897 %% Link Up: Te2/0/22
<189> DEC 18 14:22:01 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 896 %% Link Up: Te2/0/21
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 895 %% Te2/0/20 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 894 %% Te2/0/19 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 893 %% Te2/0/18 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 892 %% Te2/0/17 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 891 %% Te2/0/16 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 890 %% Te2/0/15 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 889 %% Te2/0/14 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 888 %% Te2/0/13 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 887 %% Te2/0/12 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 886 %% Te2/0/11 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 885 %% Te2/0/10 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:59 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 884 %% Te2/0/9 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:21:58 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 883 %% Entity Database: Configuration Changed
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 882 %% Te2/0/20 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 881 %% Te2/0/19 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 880 %% Te2/0/18 is transitioned from the Forwarding state to the Blocking state in instance 0
--More-- or (q)uit
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 879 %% Te2/0/17 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 878 %% Te2/0/16 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 877 %% Te2/0/15 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 876 %% Te2/0/14 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 875 %% Te2/0/13 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 874 %% Te2/0/12 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 873 %% Te2/0/11 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 872 %% Te2/0/10 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 871 %% Te2/0/9 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 870 %% Te2/0/20 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 869 %% Link Up: Te2/0/7
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 868 %% Te2/0/19 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 867 %% Te2/0/18 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 866 %% Te2/0/17 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 865 %% Te2/0/16 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 864 %% Te2/0/15 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 863 %% Te2/0/14 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 862 %% Te2/0/13 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 861 %% Te2/0/12 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 860 %% Te2/0/11 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 859 %% Link Up: Te2/0/8
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 858 %% Link Up: Te2/0/6
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 857 %% Te2/0/10 is transitioned from the Forwarding state to the Blocking state in instance 0
--More-- or (q)uit
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 856 %% Link Up: Te2/0/20
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 855 %% Link Up: Te2/0/19
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 854 %% Te2/0/9 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 853 %% Link Up: Te2/0/18
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 852 %% Link Up: Te2/0/17
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 851 %% Link Up: Te2/0/16
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 850 %% Link Up: Te2/0/15
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 849 %% Link Up: Te2/0/14
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 848 %% Link Up: Te2/0/13
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 847 %% Link Up: Te2/0/12
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 846 %% Link Up: Te2/0/11
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 845 %% Link Up: Te2/0/10
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 844 %% Link Up: Te2/0/9
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 843 %% Link Up: Te2/0/5
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 842 %% Link Up: Te2/0/2
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 841 %% Link Up: Te2/0/1
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 840 %% Link on Te2/0/20 is failed
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 839 %% Link Down: Te2/0/20
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 838 %% Link on Te2/0/19 is failed
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 837 %% Link Down: Te2/0/19
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 836 %% Link on Te2/0/18 is failed
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 835 %% Link Down: Te2/0/18
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 834 %% Link on Te2/0/17 is failed
--More-- or (q)uit
<189> DEC 18 14:21:53 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 833 %% Link Down: Te2/0/17
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 832 %% Link on Te2/0/16 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 831 %% Link Down: Te2/0/16
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 830 %% Link on Te2/0/15 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 829 %% Link Down: Te2/0/15
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 828 %% Link on Te2/0/14 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 827 %% Link Down: Te2/0/14
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 826 %% Link on Te2/0/13 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 825 %% Link Down: Te2/0/13
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 824 %% Te2/0/14 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 823 %% Link on Te2/0/12 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 822 %% Link Down: Te2/0/12
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 821 %% Link on Te2/0/11 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 820 %% Link Down: Te2/0/11
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 819 %% Link on Te2/0/10 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 818 %% Link Down: Te2/0/10
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 817 %% Link on Te2/0/9 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 816 %% Link Down: Te2/0/9
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 815 %% Link on Te2/0/5 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 814 %% Link Down: Te2/0/5
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 813 %% Link on Te2/0/2 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 812 %% Link Down: Te2/0/2
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 811 %% Link on Te2/0/1 is failed
--More-- or (q)uit
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 810 %% Link Down: Te2/0/1
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 809 %% Te2/0/13 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 808 %% Link Up: Te2/0/5
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 807 %% Te2/0/20 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 806 %% Te2/0/19 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 805 %% Te2/0/18 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 804 %% Te2/0/13 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 803 %% Te2/0/11 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 802 %% Link Up: Te2/0/14
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 801 %% Link Up: Te2/0/13
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 800 %% Te2/0/10 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 799 %% Te2/0/9 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 798 %% Link Up: Te2/0/20
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 797 %% Link Up: Te2/0/19
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 796 %% Link Up: Te2/0/18
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 795 %% Link on Te2/0/13 is failed
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 794 %% Link Down: Te2/0/13
<189> DEC 18 14:21:52 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 793 %% Link Up: Te2/0/11
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 792 %% Te2/0/17 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 791 %% Link Up: Te2/0/10
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 790 %% Link Up: Te2/0/9
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 789 %% Te2/0/16 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 788 %% Te2/0/15 is transitioned from the Forwarding state to the Blocking state in instance 0
--More-- or (q)uit
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 787 %% Te2/0/13 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 786 %% Te2/0/12 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 785 %% Link Up: Te2/0/17
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 784 %% Link Up: Te2/0/16
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 783 %% Link Up: Te2/0/15
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 782 %% Link Up: Te2/0/13
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 781 %% Link Up: Te2/0/12
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 780 %% Link Up: Te2/0/2
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 779 %% Link Up: Te2/0/1
<190> DEC 18 14:21:49 10.0.1.254-1 CKPT[249997664]: ckpt_task.c(423) 778 %% Checkpoint operation to backup unit 2 complete.
<190> DEC 18 14:21:48 10.0.1.254-1 CKPT[249997664]: ckpt_task.c(523) 777 %% New backup manager selected, unit 2.
<189> DEC 18 14:21:48 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 776 %% Stack Port Link Up: Index: 203 Unit: 2 Tag: 0/04
<189> DEC 18 14:21:48 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 775 %% Stack Port Link Up: Index: 202 Unit: 2 Tag: 0/03
<189> DEC 18 14:21:47 10.0.1.254-1 General[228876048]: sdm_template_mgr.c(577) 774 %% SDM template manager failed to propagate the next template ID to unit 2.
<188> DEC 18 14:21:47 10.0.1.254-1 DOT1S[220713824]: dot1s_transport.c(256) 773 %% Msg Send failed for port 145
<188> DEC 18 14:21:47 10.0.1.254-1 SIM[220713824]: ms_api.c(488) 772 %% msMsgSend: message transmission to unit 2, failed
<188> DEC 18 14:21:47 10.0.1.254-1 DOT1S[220713824]: dot1s_transport.c(256) 771 %% Msg Send failed for port 146
<188> DEC 18 14:21:47 10.0.1.254-1 SIM[220713824]: ms_api.c(488) 770 %% msMsgSend: message transmission to unit 2, failed
<189> DEC 18 14:21:47 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 769 %% Unit 2 is added to the stack
<189> DEC 18 14:21:28 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 768 %% Stack Port Link Up: Index: 103 Unit: 1 Tag: 0/04
<189> DEC 18 14:21:28 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 767 %% Stack Port Link Up: Index: 102 Unit: 1 Tag: 0/03
<189> DEC 18 14:21:15 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 766 %% Unit 1 identified a link failure in the stack. 0/03
<189> DEC 18 14:21:15 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 765 %% Stack Port Link Down: Index: 102 Unit: 1 Tag: 0/03
--More-- or (q)uit
<189> DEC 18 14:21:15 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 764 %% Stack Port Link Up: Index: 102 Unit: 1 Tag: 0/03
<189> DEC 18 14:21:14 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 763 %% Unit 1 identified a link failure in the stack. 0/04
<189> DEC 18 14:21:14 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 762 %% Stack Port Link Down: Index: 103 Unit: 1 Tag: 0/04
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 761 %% Te1/0/22 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 760 %% Te1/0/22 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 759 %% Link Up: Te1/0/22
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 758 %% Te1/0/24 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 757 %% Te1/0/24 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 756 %% Link Up: Te1/0/24
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 755 %% Link Up: Te1/0/21
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 754 %% Entity Database: Configuration Changed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 753 %% Po2 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 752 %% Po1 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 751 %% Te1/0/20 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 750 %% Te1/0/19 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 749 %% Te1/0/18 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 748 %% Te1/0/17 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 747 %% Te1/0/16 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 746 %% Te1/0/15 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 745 %% Te1/0/14 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 744 %% Te1/0/13 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 743 %% Te1/0/12 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 742 %% Te1/0/11 is transitioned from the Learning state to the Forwarding state in instance 0
--More-- or (q)uit
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 741 %% Te1/0/10 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 740 %% Te1/0/9 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 739 %% Te1/0/24 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 738 %% Link on Te1/0/24 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 737 %% Link Down: Te1/0/24
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 736 %% Te1/0/22 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 735 %% Link on Te1/0/22 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 734 %% Link Down: Te1/0/22
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 733 %% Link on Te1/0/21 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 732 %% Link Down: Te1/0/21
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 731 %% Warm Auto-Restart has completed on unit 1.
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 730 %% Po3 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 729 %% Instance 0 has elected a new STP root: 4000:d067:e56f:7703
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 728 %% Link Up: Vl3
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 727 %% Link Up: Vl2
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 726 %% Link Up: Vl305
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 725 %% Link Up: Vl99
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 724 %% Link Up: Vl30
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 723 %% Link Up: Vl21
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 722 %% Link Up: Vl20
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 721 %% Link Up: Vl10
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 720 %% Link Up: Vl5
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 719 %% Link Up: Vl1
--More-- or (q)uit
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 718 %% Po3 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 717 %% Po2 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 716 %% Po1 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 715 %% Te1/0/24 is transitioned from the Learning state to the Forwarding state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 714 %% Te1/0/24 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 713 %% Te1/0/22 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 712 %% Te1/0/20 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 711 %% Te1/0/19 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 710 %% Te1/0/18 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 709 %% Te1/0/17 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 708 %% Te1/0/16 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 707 %% Te1/0/15 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 706 %% Te1/0/14 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 705 %% Te1/0/13 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 704 %% Te1/0/12 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 703 %% Te1/0/11 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 702 %% Te1/0/10 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 701 %% Te1/0/9 is transitioned from the Forwarding state to the Blocking state in instance 0
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 700 %% Link Up: Te1/0/24
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 699 %% Link Up: Te1/0/24
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 698 %% Link Up: Te1/0/22
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 697 %% Link Up: Te1/0/22
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 696 %% Link Up: Te1/0/21
--More-- or (q)uit
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 695 %% Link Up: Te1/0/21
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 694 %% Link Up: Te1/0/20
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 693 %% Link Up: Te1/0/20
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 692 %% Link Up: Te1/0/19
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 691 %% Link Up: Te1/0/19
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 690 %% Link Up: Te1/0/18
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 689 %% Link Up: Te1/0/18
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 688 %% Link Up: Te1/0/17
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 687 %% Link Up: Te1/0/17
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 686 %% Link Up: Te1/0/16
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 685 %% Link Up: Te1/0/16
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 684 %% Link Up: Te1/0/15
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 683 %% Link Up: Te1/0/15
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 682 %% Link Up: Te1/0/14
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 681 %% Link Up: Te1/0/14
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 680 %% Link Up: Te1/0/13
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 679 %% Link Up: Te1/0/13
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 678 %% Link Up: Te1/0/12
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 677 %% Link Up: Te1/0/24
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 676 %% Link Up: Te1/0/12
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 675 %% Link Up: Te1/0/11
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 674 %% Link Up: Te1/0/11
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 673 %% Link Up: Te1/0/22
--More-- or (q)uit
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 672 %% Link Up: Te1/0/10
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 671 %% Link Up: Te1/0/21
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 670 %% Link Up: Te1/0/10
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 669 %% Link Up: Te1/0/9
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 668 %% Link Up: Te1/0/9
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 667 %% Link Up: Te1/0/20
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 666 %% Link Up: Te1/0/8
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 665 %% Link Up: Te1/0/19
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 664 %% Link Up: Te1/0/8
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 663 %% Link Up: Te1/0/18
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 662 %% Link Up: Te1/0/7
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 661 %% Link Up: Te1/0/7
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 660 %% Link Up: Te1/0/17
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 659 %% Link Up: Te1/0/6
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 658 %% Link Up: Te1/0/6
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 657 %% Link Up: Te1/0/16
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 656 %% Link Up: Te1/0/5
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 655 %% Link Up: Te1/0/5
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 654 %% Link Up: Te1/0/15
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 653 %% Link Up: Te1/0/2
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 652 %% Link Up: Te1/0/2
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 651 %% Link Up: Te1/0/14
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 650 %% Link Up: Te1/0/1
--More-- or (q)uit
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 649 %% Link Up: Te1/0/13
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 648 %% Link Up: Te1/0/1
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 647 %% Link Up: Te1/0/12
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 646 %% Link Up: Te1/0/11
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 645 %% Link Up: Te1/0/10
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 644 %% Link Up: Te1/0/9
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 643 %% Link Up: Te1/0/8
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 642 %% Link Up: Te1/0/7
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 641 %% Link Up: Te1/0/6
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 640 %% Link Up: Te1/0/5
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 639 %% Link Up: Te1/0/2
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 638 %% Link Up: Te1/0/1
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 637 %% Link on Vl305 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 636 %% Link Down: Vl305
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 635 %% Link on Vl99 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 634 %% Link Down: Vl99
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 633 %% Link on Vl30 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 632 %% Link Down: Vl30
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 631 %% Link on Vl21 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 630 %% Link Down: Vl21
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 629 %% Link on Vl20 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 628 %% Link Down: Vl20
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 627 %% Link on Vl10 is failed
--More-- or (q)uit
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 626 %% Link Down: Vl10
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 625 %% Link on Vl5 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 624 %% Link Down: Vl5
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 623 %% Link on Vl3 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 622 %% Link Down: Vl3
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 621 %% Link on Vl2 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 620 %% Link Down: Vl2
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 619 %% Link on Vl1 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 618 %% Link Down: Vl1
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 617 %% Unit 2 is added to the stack
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 616 %% Stack Master unit 2 is failed
<189> DEC 18 14:20:12 10.0.1.254-1 TRAPMGR[200995696]: traputil.c(637) 615 %% Cold Start: Unit: 0
<190> DEC 18 14:19:16 10.0.1.254-1 UNITMGR[228710800]: unitmgr.c(2418) 614 %% Warm Auto-Restart complete on unit 1
<188> DEC 18 14:19:16 10.0.1.254-1 DRIVER[111653248]: l7_usl_port_db.c(8577) 613 %% Reconciliation error: Unit/Port 0/21: CmdId 10 does not have a valid value, operValid 4 shadowValid 0
<187> DEC 18 14:19:15 10.0.1.254-1 IP[250144544]: ip_util.c(3395) 610 %% ipMapRtrIntfStaticConfigApply(): ERROR: Failed to configure static route to 0.0.0.0/0.0.0.0
<190> DEC 18 14:19:14 10.0.1.254-1 UNITMGR[192111440]: unitmgr.c(6850) 606 %% No Potential unit to configure as Standby when unit 1 joined
<190> DEC 18 14:19:14 10.0.1.254-1 AUTO_INST[180488256]: auto_install_control.c(1259) 605 %% AutoInstall is stopped.
<190> DEC 18 14:19:13 10.0.1.254-1 SSHD[249586208]: sshd_control.c(549) 604 %% sshdEventAdminModeSet success, event=0
<189> DEC 18 14:19:13 10.0.1.254-1 SSHD[249586208]: sshd_control.c(226) 603 %% sshdListenTask 0x17c7e910 started, exited = 0. The SSHD listen task was properly started, also printing the value of the ▒exited▒ flag.
<190> DEC 18 14:19:13 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 601 %% [CLI:admin:EIA-232] Access level of user root has been set to 15
<190> DEC 18 14:19:13 10.0.1.254-1 CLI_WEB[180488256]: cmd_logger_api.c(260) 599 %% [CLI:admin:EIA-232] New user root has been added
<187> DEC 18 14:19:13 0.0.0.0-1 SIM[180488256]: sim_svc_port.c(428) 598 %% Failed to set the default gateway in the IP stack.
<190> DEC 18 14:19:13 0.0.0.0-1 DRIVER[217492016]: dapi.c(975) 14 %% Error on command 50: usp 2/0/0: Card Not Attached.
--More-- or (q)uit
<189> DEC 18 14:19:13 0.0.0.0-1 TRAPMGR[249765888]: traputil.c(637) 13 %% Stack Port Link Up: Index: 103 Unit: 1 Tag: 0/04
<190> DEC 18 14:19:12 0.0.0.0-1 NIM[173050720]: nim_intf_api.c(91) 10 %% NIM: incorrect phase 155 for operation
<185> DEC 18 14:19:11 0.0.0.0-1 UNITMGR[228710800]: unitmgr.c(2753) 9 %% Standby taking over as Manager of the Stack, reason: Stack Link Down
<189> DEC 18 22:06:30 0.0.0.0-1 BSP[81199760]: cpu_env_monitor.c(264) 6 %% Power Supply 1 Status : OK
<189> DEC 18 22:06:30 0.0.0.0-1 BSP[81199760]: cpu_env_monitor.c(264) 5 %% Power Supply 0 Status : OK
<189> DEC 18 22:06:30 0.0.0.0-1 BSP[81199760]: cpu_env_monitor.c(206) 4 %% Temperature Status Change, current = 2, previous = 1 FanSpeed set to: 9000
<189> DEC 18 22:06:18 0.0.0.0-1 General[1073741072]: bootos.c(242) 2 %% Starting code... BSP initialization complete, starting system application.
<186> DEC 18 22:06:18 0.0.0.0-1 General[1073741072]: bootos.c(225) 1 %% Event(0xaaaaaaaa)
Thanks alot
Latest firmware, just upgraded the other week, 5.1.9.3 from memory .. VMWare teaming mode is non LACP. We get fail over nicely when pulling a cable from a NIC port on the hypervisors, it just takes 1 ping drop for VMWare to move the VM's onto the other NIC and then away they go, but reloading a switch or rebooting fully looses connectivity with hypervisors, even tho the NIC ports from the hypervisors are spread accross the stack into different switches. EG. VMware #1 is plugged into Switch 1 and 2 , and pulling a network cable from one NIC port I see 1 ping drop, as the VMs bound to one NIC move to the other, then they start pinging again, however, if I pull a stack port on one of the switches be it master or slave, I see a full loss of connectivity to the hypervisors. Its kind of abit difficult to keep trying to re-enact the scenario being that these machines are in production, tho over the Xmas break I managed to try every cobination thinkable, e.g reload switches by themselves, and pulling out Stacking cables, and I saw similar unexpected behaviour issuse. The logs earlier were from pulling a stacking port, just one, tho I cant remember if it was from the master of slave
Thanks for your time
show switch stack-ports :
Configured Running
Stack Stack Link Link
Interface Mode Mode Status Speed (Gb/s)
---------------- ---------- ---------- ------------ ------------
Te1/0/1 Ethernet Ethernet Link Up 10
Te1/0/2 Ethernet Ethernet Link Up 10
Te1/0/3 Stack Stack Link Up 10
Te1/0/4 Stack Stack Link Up 10
Te1/0/5 Ethernet Ethernet Link Up 10
Te1/0/6 Ethernet Ethernet Link Up 10
Te1/0/7 Ethernet Ethernet Link Up 10
Te1/0/8 Ethernet Ethernet Link Up 10
Te1/0/9 Ethernet Ethernet Link Up 10
Te1/0/10 Ethernet Ethernet Link Up 10
Te1/0/11 Ethernet Ethernet Link Up 10
Te1/0/12 Ethernet Ethernet Link Up 10
Te1/0/13 Ethernet Ethernet Link Up 10
Te1/0/14 Ethernet Ethernet Link Up 10
Te1/0/15 Ethernet Ethernet Link Up 10
Te1/0/16 Ethernet Ethernet Link Up 10
Te1/0/17 Ethernet Ethernet Link Up 10
Te1/0/18 Ethernet Ethernet Link Up 10
--More-- or (q)uit
Te1/0/19 Ethernet Ethernet Link Up 10
Te1/0/20 Ethernet Ethernet Link Up 10
Te1/0/21 Ethernet Ethernet Link Up 10
Te1/0/22 Ethernet Ethernet Link Up 1
Te1/0/23 Ethernet Ethernet Link Down Unknown
Te1/0/24 Ethernet Ethernet Link Up 10
Te2/0/1 Ethernet Ethernet Link Up 10
Te2/0/2 Ethernet Ethernet Link Up 10
Te2/0/3 Stack Stack Link Up 10
Te2/0/4 Stack Stack Link Up 10
Te2/0/5 Ethernet Ethernet Link Up 10
Te2/0/6 Ethernet Ethernet Link Up 10
Te2/0/7 Ethernet Ethernet Link Up 10
Te2/0/8 Ethernet Ethernet Link Up 10
Te2/0/9 Ethernet Ethernet Link Up 10
Te2/0/10 Ethernet Ethernet Link Up 10
Te2/0/11 Ethernet Ethernet Link Up 10
Te2/0/12 Ethernet Ethernet Link Up 10
Te2/0/13 Ethernet Ethernet Link Up 10
Te2/0/14 Ethernet Ethernet Link Up 10
Te2/0/15 Ethernet Ethernet Link Up 10
Te2/0/16 Ethernet Ethernet Link Up 10
Te2/0/17 Ethernet Ethernet Link Up 10
--More-- or (q)uit
Te2/0/18 Ethernet Ethernet Link Up 10
Te2/0/19 Ethernet Ethernet Link Up 10
Te2/0/20 Ethernet Ethernet Link Up 10
Te2/0/21 Ethernet Ethernet Link Up 10
Te2/0/22 Ethernet Ethernet Link Up 1
Te2/0/23 Ethernet Ethernet Link Down Unknown
Te2/0/24 Ethernet Ethernet Link Up 10
Te3/0/1 Ethernet Ethernet Link Down Unknown
Te3/0/2 Ethernet Ethernet Link Down Unknown
Te3/0/3 Stack Stack Link Up 10
Te3/0/4 Stack Stack Link Up 10
Te3/0/5 Ethernet Ethernet Link Up 10
Te3/0/6 Ethernet Ethernet Link Up 10
Te3/0/7 Ethernet Ethernet Link Down Unknown
Te3/0/8 Ethernet Ethernet Link Down Unknown
Te3/0/9 Ethernet Ethernet Link Down Unknown
Te3/0/10 Ethernet Ethernet Link Down Unknown
Te3/0/11 Ethernet Ethernet Link Down Unknown
Te3/0/12 Ethernet Ethernet Link Down Unknown
Te3/0/13 Ethernet Ethernet Link Down Unknown
Te3/0/14 Ethernet Ethernet Link Down Unknown
Te3/0/15 Ethernet Ethernet Link Down Unknown
Te3/0/16 Ethernet Ethernet Link Down Unknown
--More-- or (q)uit
Te3/0/17 Ethernet Ethernet Link Down Unknown
Te3/0/18 Ethernet Ethernet Link Down Unknown
Te3/0/19 Stack Stack Link Down 10
Te3/0/20 Stack Stack Link Down 10
Te3/0/21 Ethernet Ethernet Link Down Unknown
Te3/0/22 Ethernet Ethernet Link Down Unknown
Te3/0/23 Ethernet Ethernet Link Up 10
Te3/0/24 Ethernet Ethernet Link Up 10stack port counters
OFF-STK-01#show switch stack-ports counters
------------TX-------------- ------------RX--------------
Data Error Data Error
Rate Rate Total Rate Rate Total
Interface (Mb/s) (Errors/s) Errors (Mb/s) (Errors/s) Errors
---------------- ------ ---------- ---------- ------ ---------- ----------
Te1/0/1 0 0 0 0 0 0
Te1/0/2 0 0 0 0 0 0
Te1/0/3 0 0 0 0 0 0
Te1/0/4 57 0 0 548 0 0
Te1/0/5 0 0 0 0 0 0
Te1/0/6 0 0 0 0 0 0
Te1/0/7 0 0 0 0 0 0
Te1/0/8 0 0 0 0 0 0
Te1/0/9 0 0 0 0 0 0
Te1/0/10 0 0 0 0 0 0
Te1/0/11 0 0 0 0 0 0
Te1/0/12 0 0 0 0 0 0
Te1/0/13 0 0 0 0 0 0
Te1/0/14 0 0 0 0 0 0
Te1/0/15 0 0 0 0 0 0
Te1/0/16 0 0 0 0 0 0
Te1/0/17 0 0 0 0 0 0
--More-- or (q)uit
Te1/0/18 0 0 0 0 0 0
Te1/0/19 0 0 0 0 0 0
Te1/0/20 0 0 0 0 0 0
Te1/0/21 0 0 0 0 0 0
Te1/0/22 0 0 0 0 0 0
Te1/0/23 0 0 0 0 0 0
Te1/0/24 0 0 0 0 0 0
Te2/0/1 0 0 0 0 0 0
Te2/0/2 0 0 0 0 0 0
Te2/0/3 0 0 0 0 0 0
Te2/0/4 555 0 0 56 0 0
Te2/0/5 0 0 0 0 0 0
Te2/0/6 0 0 0 0 0 0
Te2/0/7 0 0 0 0 0 0
Te2/0/8 0 0 0 0 0 0
Te2/0/9 0 0 0 0 0 0
Te2/0/10 0 0 0 0 0 0
Te2/0/11 0 0 0 0 0 0
Te2/0/12 0 0 0 0 0 0
Te2/0/13 0 0 0 0 0 0
Te2/0/14 0 0 0 0 0 0
Te2/0/15 0 0 0 0 0 0
Te2/0/16 0 0 0 0 0 0
--More-- or (q)uit
Te2/0/17 0 0 0 0 0 0
Te2/0/18 0 0 0 0 0 0
Te2/0/19 0 0 0 0 0 0
Te2/0/20 0 0 0 0 0 0
Te2/0/21 0 0 0 0 0 0
Te2/0/22 0 0 0 0 0 0
Te2/0/23 0 0 0 0 0 0
Te2/0/24 0 0 0 0 0 0
Te3/0/1 0 0 0 0 0 0
Te3/0/2 0 0 0 0 0 0
Te3/0/3 0 0 0 0 0 0
Te3/0/4 0 0 0 0 0 0
Te3/0/5 0 0 0 0 0 0
Te3/0/6 0 0 0 0 0 0
Te3/0/7 0 0 0 0 0 0
Te3/0/8 0 0 0 0 0 0
Te3/0/9 0 0 0 0 0 0
Te3/0/10 0 0 0 0 0 0
Te3/0/11 0 0 0 0 0 0
Te3/0/12 0 0 0 0 0 0
Te3/0/13 0 0 0 0 0 0
Te3/0/14 0 0 0 0 0 0
Te3/0/15 0 0 0 0 0 0
--More-- or (q)uit
Te3/0/16 0 0 0 0 0 0
Te3/0/17 0 0 0 0 0 0
Te3/0/18 0 0 0 0 0 0
Te3/0/19 0 0 0 0 0 0
Te3/0/20 0 0 0 0 0 0
Te3/0/21 0 0 0 0 0 0
Te3/0/22 0 0 0 0 0 0
Te3/0/23 0 0 0 0 0 0
Te3/0/24 0 0 0 0 0 0
zzzp87
3 Posts
2367
0
Posted January 9th, 2017 06:00
Thanks alot
When rebooting stack members individually, we also get an outage from devices using that switch..
Specfially tho in the logs , this part here to me tho I dont really understand, looks abit nasty, and I suspect this may be the issue to the fail over not working, tho I dont quite understand what it means. Do you think it could be related? This log was generated by me pulling out an individual stacking port .
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 780 %% Link Up: Te2/0/2
<189> DEC 18 14:21:51 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 779 %% Link Up: Te2/0/1
<190> DEC 18 14:21:49 10.0.1.254-1 CKPT[249997664]: ckpt_task.c(423) 778 %% Checkpoint operation to backup unit 2 complete.
<190> DEC 18 14:21:48 10.0.1.254-1 CKPT[249997664]: ckpt_task.c(523) 777 %% New backup manager selected, unit 2.
<189> DEC 18 14:21:48 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 776 %% Stack Port Link Up: Index: 203 Unit: 2 Tag: 0/04
<189> DEC 18 14:21:48 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 775 %% Stack Port Link Up: Index: 202 Unit: 2 Tag: 0/03
<189> DEC 18 14:21:47 10.0.1.254-1 General[228876048]: sdm_template_mgr.c(577) 774 %% SDM template manager failed to propagate the next template ID to unit 2.
<188> DEC 18 14:21:47 10.0.1.254-1 DOT1S[220713824]: dot1s_transport.c(256) 773 %% Msg Send failed for port 145
<188> DEC 18 14:21:47 10.0.1.254-1 SIM[220713824]: ms_api.c(488) 772 %% msMsgSend: message transmission to unit 2, failed
<188> DEC 18 14:21:47 10.0.1.254-1 DOT1S[220713824]: dot1s_transport.c(256) 771 %% Msg Send failed for port 146
<188> DEC 18 14:21:47 10.0.1.254-1 SIM[220713824]: ms_api.c(488) 770 %% msMsgSend: message transmission to unit 2, failed
<189> DEC 18 14:21:47 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 769 %% Unit 2 is added to the stack
<189> DEC 18 14:21:28 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 768 %% Stack Port Link Up: Index: 103 Unit: 1 Tag: 0/04
<189> DEC 18 14:21:28 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 767 %% Stack Port Link Up: Index: 102 Unit: 1 Tag: 0/03
<189> DEC 18 14:21:15 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 766 %% Unit 1 identified a link failure in the stack. 0/03
<189> DEC 18 14:21:15 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 765 %% Stack Port Link Down: Index: 102 Unit: 1 Tag: 0/03
--More-- or (q)uit
<189> DEC 18 14:21:15 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 764 %% Stack Port Link Up: Index: 102 Unit: 1 Tag: 0/03
<189> DEC 18 14:21:14 10.0.1.254-1 TRAPMGR[246854944]: traputil.c(637) 763 %% Unit 1 identified a link failure in the stack. 0/04
<189> DEC 18 14:21:14 10.0.1.254-1 TRAPMGR[249765888]: traputil.c(637) 762 %% Stack Port Link Down: Index: 103 Unit: 1 Tag: 0/04