UNSOLVED

zhukk

updated

4 years ago

Z

zhukk

1 Rookie

21 Posts

0

2055

September 20th, 2022 07:00

drop packets s4148f

After replacing the switch with a dell s4148, I had complaints about short-term ping drops, torn RDP sessions and problems in games.
Build Version: 10.5.3.0.44
Build Time: 2021-10-06T23:03:55+0000
System Type: S4148F-ON

I use it as a router with BGP.
tracing causes packet loss on the next hop after the router. I can't figure out what is the reason? on other models (s5148f) that are and are similarly configured, there are no such problems. The only thing that indicates problems is discarded packets on outgoing statistics?
Where do these packages come from and whether they can affect my problems. Because of what else can there be such symptoms with packet loss.

for example 

 

 

Output statistics:
     83575500834 packets, 99634045902720 octets
     474752979 64-byte pkts, 9371467670 over 64-byte pkts, 2420655182 over 127-byte pkts
     1388840078 over 255-byte pkts, 2279175936 over 511-byte pkts, 6.7640608989e+10 over 1023-byte pkts
     2226281 Multicasts, 171032122 Broadcasts, 83402242466 Unicasts
     0 throttles, 11972 discarded, 0 Collisions,  wred drops
Rate Info(interval 30 seconds):

 

 

  • DELL-Chris H

    7 Practitioner

    9682 Posts

    48046 Points

    647

    0

    Posted September 20th, 2022 13:00

    Zhukk,

     

    Would you be able to confirm a few things for me?

     

    Are the client devices directly connected,  or are they going through other switches?

     

    If the clients are not directly connected, are you able to test with one to see if the issue still occurs if directly connected?

     

    Is the switch up to date on firmware?

     

    Would you be able to upload the show tech support results to upload . dell. com and private message me the svc tag used to upload them, so that I can locate them?

     

    Let me know and we can go from there.

     

     

    DELL-Chris H

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’. ‘Thumbs up’ the posts you like!

  • zhukk

    1 Rookie

    21 Posts

    633

    0

    Posted September 21st, 2022 00:00

    anwer in PM

  • DELL-Chris H

    7 Practitioner

    9682 Posts

    48046 Points

    614

    0

    Posted September 21st, 2022 06:00

    I have recieved the logs, let me review and do some research and I will be back with you as sonn as possible.

     

    Thanks.

     

    DELL-Chris H

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’. ‘Thumbs up’ the posts you like!

  • DELL-Chris H

    7 Practitioner

    9682 Posts

    48046 Points

    612

    0

    Posted September 21st, 2022 07:00

    Zhukk,

     

    Since the counters are accumulative, would you please;

     

    First: clear counters

    wait 5 minutes

     

    show interface | grep Ethernet|throttles|discarded | no-more

     

    system "sudo hshell -c 'ps'"

     

    system "sudo hshell -c 'show c'"  wait 5 seconds and repeat this last command 5 times.

     

    This way we can see if discards are still happening and look for a cause.

     

     

     

    Also, by the "show interfaces" we can see a lot of traffic being received by the switch on interfaces:

     

    Line 1153: Ethernet 1/1/33 is up, line protocol is up
    Line 1168: Queuing strategy: fifo
    Line 1174: 0 runts, 0 giants, 0 throttles
    Line 1175: 0 CRC, 0 overrun, 0 discarded
    Line 1181: 0 throttles, 1 discarded, 0 Collisions, wred drops
    Line 1182: Rate Info(interval 30 seconds):
    Line 1183: Input 2530 Mbits/sec, 236574 packets/sec, 25% of line rate
    Line 1184: Output 799 Mbits/sec, 122725 packets/sec, 7% of line rate
    Line 1188: Ethernet 1/1/34 is up, line protocol is up
    Line 1203: Queuing strategy: fifo
    Line 1209: 0 runts, 0 giants, 0 throttles
    Line 1210: 0 CRC, 0 overrun, 0 discarded
    Line 1216: 0 throttles, 284 discarded, 0 Collisions, wred drops
    Line 1217: Rate Info(interval 30 seconds):
    Line 1218: Input 2525 Mbits/sec, 234086 packets/sec, 25% of line rate
    Line 1219: Output 808 Mbits/sec, 122388 packets/sec, 8% of line rate
    Line 1223: Ethernet 1/1/35 is up, line protocol is up
    Line 1238: Queuing strategy: fifo
    Line 1244: 0 runts, 0 giants, 0 throttles
    Line 1245: 0 CRC, 0 overrun, 0 discarded
    Line 1251: 0 throttles, 369 discarded, 0 Collisions, wred drops
    Line 1252: Rate Info(interval 30 seconds):
    Line 1253: Input 2649 Mbits/sec, 245393 packets/sec, 26% of line rate
    Line 1254: Output 744 Mbits/sec, 118145 packets/sec, 7% of line rate

    Line 1685: Ethernet 1/1/48 is up, line protocol is up
    Line 1700: Queuing strategy: fifo
    Line 1706: 0 runts, 0 giants, 0 throttles
    Line 1707: 2 CRC, 0 overrun, 2 discarded
    Line 1713: 0 throttles, 24862 discarded, 0 Collisions, wred drops
    Line 1714: Rate Info(interval 30 seconds):
    Line 1715: Input 4782 Mbits/sec, 469196 packets/sec, 47% of line rate
    Line 1716: Output 624 Mbits/sec, 133406 packets/sec, 6% of line rate
    Line 1720: Ethernet 1/1/49 is up, line protocol is up
    Line 1736: Queuing strategy: fifo
    Line 1742: 0 runts, 0 giants, 0 throttles
    Line 1743: 0 CRC, 0 overrun, 0 discarded
    Line 1749: 0 throttles, 14489 discarded, 0 Collisions, wred drops
    Line 1750: Rate Info(interval 30 seconds):
    Line 1751: Input 3501 Mbits/sec, 403940 packets/sec, 35% of line rate
    Line 1752: Output 1040 Mbits/sec, 221537 packets/sec, 10% of line rate
    Line 1756: Ethernet 1/1/50 is up, line protocol is up
    Line 1772: Queuing strategy: fifo
    Line 1778: 0 runts, 0 giants, 0 throttles
    Line 1779: 1 CRC, 0 overrun, 1 discarded
    Line 1785: 0 throttles, 52291 discarded, 0 Collisions, wred drops
    Line 1786: Rate Info(interval 30 seconds):
    Line 1787: Input 3713 Mbits/sec, 419386 packets/sec, 37% of line rate
    Line 1788: Output 1174 Mbits/sec, 236952 packets/sec, 11% of line rate
    Line 1792: Ethernet 1/1/51 is up, line protocol is up
    Line 1808: Queuing strategy: fifo
    Line 1814: 0 runts, 0 giants, 0 throttles
    Line 1815: 1 CRC, 0 overrun, 1 discarded
    Line 1821: 0 throttles, 32316 discarded, 0 Collisions, wred drops
    Line 1822: Rate Info(interval 30 seconds):
    Line 1823: Input 3628 Mbits/sec, 414943 packets/sec, 36% of line rate
    Line 1824: Output 1033 Mbits/sec, 223611 packets/sec, 10% of line rate

     

    Check if this is normal or maybe we have some looping in the network. These are the devices connected to these interfaces:


    ----------------------------------- show lldp neighbors -------------------

    Loc PortID Rem Host Name Rem Port Id Rem Chassis Id
    --------------------------------------------------------------------------------------
    ethernet1/1/33 triolan-plv101 eth0 9NQ5NN2
    ethernet1/1/34 triolan-plv102 eth0 JD67JN2
    ethernet1/1/35 triolan-plv103 eth0 27QWHP2
    thernet1/1/48 10.1.0.10 TenGigabitEthernet 0/46 00:01:e8:d7:b0:b2

    and:
    interface ethernet1/1/48
    description ETT_Kharkov
    no shutdown
    switchport mode trunk
    switchport trunk allowed vlan 11,85,591,789,1500,2495
    flowcontrol receive on
    !
    interface ethernet1/1/49
    description Kyiv_30g
    no shutdown
    channel-group 2 mode active
    no switchport
    flowcontrol receive on
    !
    interface ethernet1/1/50
    description Kyiv_30g
    no shutdown
    channel-group 2 mode active
    no switchport
    flowcontrol receive on
    !
    interface ethernet1/1/51
    description Kyiv_30g
    no shutdown
    channel-group 2 mode active
    no switchport
    flowcontrol receive on

    DELL-Chris H

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’. ‘Thumbs up’ the posts you like!

  • zhukk

    1 Rookie

    21 Posts

    600

    0

    Posted September 21st, 2022 12:00

    i send informaition what you need. 

    its normal traffic on this interface.

    and this device are connected to th switch, its GGC servers.

    and I also noticed that most of the lost packets were on gig links. I moved them to another switch. but I still have losses on the interfaces. an average of 0.1-0.2% in some directions. and it doesn't happen all the time, but sometimes every 5-10 minutes or so a few packets will be lost.

  • zhukk

    1 Rookie

    21 Posts

    586

    0

    Posted September 21st, 2022 23:00

    i upload additional log right after drop

    and i update firmware 

  • zhukk

    1 Rookie

    21 Posts

    583

    0

    Posted September 22nd, 2022 02:00

    i have one more question about s4148f and UTF modes.

    in manual 

    zhukk_0-1663839166066.png

    in switch

    zhukk_1-1663839207072.png

    default mode is different 

    how i can make utf mode such as manual?

  • DELL-Chris H

    7 Practitioner

    9682 Posts

    48046 Points

    578

    0

    Posted September 22nd, 2022 05:00

    Zhukk,

     

     

    This is the relevant information I found:

     

    Interfaces xe30 (ethernet 1/1/37) and xe41 (ethernet 1/1/48) are presenting drops due to receiving packets with a VLAN that is not configured on them:


    "RDBGC8.xe30 " " 877,903,414 " 439,948 " " 821/s
    "RDBGC8.xe30 " " 877,911,862 " 8,448 " " 835/s 8,448 835/s
    "RDBGC8.xe30 " " 877,925,583 " 13,721 " " 771/s 13,721 771/s
    "RDBGC8.xe30 " " 877,934,943 " 9,360 " " 966/s 9,360 966/s
    "RDBGC8.xe30 " " 877,939,532 " 4,589 " " 825/s 4,589 825/s
    "RDBGC8.xe30 " " 878,098,347 " 158,815 " " 1,106/s 158,815 1,106/s
    "RDBGC8.xe30 " " 878,108,820 " 10,473 " " 794/s 10,473 794/s
    "RDBGC8.xe30 " " 878,113,578 " 4,758 " " 891/s 4,758 891/s
    "RDBGC8.xe30 " " 878,121,677 " 8,099 " " 721/s 8,099 721/s
    "RDBGC8.xe30 " " 878,125,801 " 4,124 " " 775/s 4,124

    "RDBGC8.xe41 " " 219,263,255,399 " 69,505,458 " 131,358/s"
    "RDBGC8.xe41 " " 219,264,660,093 " 1,404,694 " 131,883/s" 131,883/s
    "RDBGC8.xe41 " " 219,266,785,111 " 2,125,018 " 131,043/s" 131,043/s
    "RDBGC8.xe41 " " 219,268,203,872 " 1,418,761 " 131,842/s" 131,842/s
    "RDBGC8.xe41 " " 219,268,916,618 " 712,746 " 128,089/s" 128,089/s
    "RDBGC8.xe41 " " 219,294,115,501 " 25,198,883 " 131,230/s" 131,230/s
    "RDBGC8.xe41 " " 219,295,522,763 " 1,407,262 " 132,340/s" 132,340/s
    "RDBGC8.xe41 " " 219,296,219,846 " 697,083 " 130,503/s" 130,503/s
    "RDBGC8.xe41 " " 219,297,617,531 " 1,397,685 " 129,266/s" 129,266/s
    "RDBGC8.xe41 " " 219,298,313,191 " 695,660 " 130,813/s" 130,813/s

     

    Other interfaces are also presenting drops but these two are the worst.

     

    Please check the other device connected to these interfaces to see if they have the same vlan configuration:

     

    interface ethernet1/1/37
    description 97_sammit
    no shutdown
    switchport mode trunk
    switchport trunk allowed vlan 95,195,500,1548,1564,1566,1678
    flowcontrol receive on

     

    interface ethernet1/1/48
    description ETT_Kharkov
    no shutdown
    switchport mode trunk
    switchport trunk allowed vlan 11,85,591,789,1500,2495
    flowcontrol receive on


    Also, we can see lots of packets loss to the CPU queue 6. This queue is reserved for ARP Request services.

    "MCQ_DROP_PKT(6).cpu0 " " 1,695,580,208 " 686,654
    "MCQ_DROP_PKT(6).cpu0 " " 1,695,593,628 " 13,420
    "MCQ_DROP_PKT(6).cpu0 " " 1,695,616,436 " 22,808
    "MCQ_DROP_PKT(6).cpu0 " " 1,695,622,737 " 6,301
    "MCQ_DROP_PKT(6).cpu0 " " 1,695,636,647 " 13,910
    "MCQ_DROP_PKT(6).cpu0 " " 1,695,874,071 " 237,424
    "MCQ_DROP_PKT(6).cpu0 " " 1,695,889,420 " 15,349
    "MCQ_DROP_PKT(6).cpu0 " " 1,695,895,983 " 6,563
    "MCQ_DROP_PKT(6).cpu0 " " 1,695,909,577 " 13,594
    "MCQ_DROP_PKT(6).cpu0 " " 1,695,915,414 " 5,837

     

    This may cause intermittent connectivity problems to your users.

     

    Please send me:

    show control-plane statistics
    show control-plane info
    show control-plane buffers
    show control-plane buffer-stats

     

    Depending on this we may have to collect a TCPDUMP on the switch, but we will see later.

     

    Thanks.

    DELL-Chris H

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’. ‘Thumbs up’ the posts you like!

  • zhukk

    1 Rookie

    21 Posts

    574

    0

    Posted September 22nd, 2022 06:00

    i send information what you asked.

    about vlan on Interfaces xe30 (ethernet 1/1/37) and xe41 (ethernet 1/1/48) its normal for this ports. this is done in case of an accident on the optical lines. if this can affect the performance of the switch, then I will remove the extra vlans.

    what i can do with packets loss to the CPU queue 6?
    and my question about UTF mode?

  • DELL-Chris H

    7 Practitioner

    9682 Posts

    48046 Points

    222

    0

    Posted September 22nd, 2022 07:00

    Sorry for the delay,

    Normally if you wanted to have us remote you would need to call in to the phone queue, but I do have some additional information after testing.

    What I would suggest is to delete the vlans not in use from the interfaces. It is spending resources receiving unnecessary traffic, looking into internal tables and discarding them after all.

    Regarding drops on cpu queue 6, it can be caused by devices flooding ARP requests or other traffic into the switch.

    You can capture the traffic that goes to the switch cpu using:

    S4148F#
    S4148F# system "sudo -i"
    [sudo] password for admin:
    root@S4148F:~# tcpdump -i any -vvv -w ./test.pcap
    tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes

    Use CTRL-C to stop, after that move the file to /home/admin directory:
    root@S4148F:~# ls -l
    total 232
    -r-------- 1 root root 188363 Aug 12 09:39 install.log
    -rw-r--r-- 1 root root 22 Aug 12 09:39 install.status
    -rw-r--r-- 1 root root 38831 Sep 22 22:44 test.pcap
    root@S4148F:~# pwd
    /root
    root@S4148F:~# mv ./test.pcap /home/admin/
    root@S4148F:~# cd /home/admin
    root@S4148F:/home/admin# ls -l
    total 48
    -rw-r--r-- 1 root root 38831 Sep 22 22:44 test.pcap
    root@S4148F:/home/admin#
    root@S4148F:/home/admin#
    root@S4148F:/home/admin# exit
    logout
    S4148F#

    From the switch cli you can grab the file using a FTP, TFTP, SCP server:
    S4148F# dir home

    Directory contents for folder: home
    Date (modified) Size (bytes) Name
    --------------------- ------------ ------------------------------------------
    2022-09-22T22:44:21Z 38831 test.pcap
    S4148F#

    S4148F# copy home://test.pcap ?
    config: Copy to configuration directory (config://filepath)
    ftp: Copy to remote FTP server (ftp://userid:passwd@hostip/filepath)
    home: Copy to home directory (home://filepath)
    scp: Copy to remote SCP server (scp://userid:passwd@hostip/filepath)
    sftp: Copy to remote SFTP server (sftp://userid:passwd@hostip/filepath)
    tftp: Copy to remote TFTP server (tftp://hostip/filepath)
    http: Copy to remote HTTP server (http://hostip/filepath)
    https: Copy to remote HTTPS server (https://hostip/filepath)
    usb: Copy to USB filesystem (usb://filepath)
    userservices: Copy to Userservices directory (userservices://filepath)
    running-configuration Copy to running-configuration
    severity-profile: Copy to severity-profile directory (severity-profile://filepath)

    S4148F# copy home://test.pcap ftp://userid:passwd@hostip/filepath

    Now in regard to UFT, it looks like it might be an error in our documentation. I tested on one of the S4148F-ON in our lab and had the same result:

    S4148F# show hardware forwarding-table mode all
    Mode default scaled-l2 scaled-l3-routes scaled-l3-hosts
    L2 MAC Entries 147456 278528 16384 16384
    L3 Host Entries 135168 4096 4096 266240
    L3 Route Entries 16384 16384 262144 16384

     

    Let me know if this helps.

    DELL-Chris H

    Social Media and Communities Professional

    Dell Technologies | Enterprise Support Services

    #IWork4Dell

    Did I answer your query? Please click on ‘Mark as Accepted Answer’. ‘Thumbs up’ the posts you like!