Dell Networking OS10 Routes are not being installed

• Collecting Information:
  • Use the sosreport command to collect system logs and hardware tables information.
  • Collect topology maps and configuration details.
  • hshell -c "l3 info" command shows if the table space is exhausted.
• Identifying the Issue:
  • Check if which routes are not working and the devices are unable to communicate.
  • Using OS10 show commands, you can see the routes in the routing table.
  • From the OS10 Kernel, ping works between the source and the destination.
  • Using the hshell command, the route can not seen in the defip table.
    • For IPv4, hshell -c "´l3 defip show" can be used to check the route entry.
    • For IPv6, hshell -c "l3 ip6route show" can be used to check the route entry.
    • Check the current table's usage with hshell -c "l3 info" command.
  • You can also pull the journalctl output looking for the entries below.

//========

[ROUTE:HAL-RT-DR], Error: hal_fib_route_add vrf_id: 8, prefix: 201.54.14.249, prefix_len: 32, hal_err: -1 (Failure)
base_nas[err]:  [ev_log_t_SAI_ROUTE:Switch Id: 0], Failed to add route in Unit 0. Err: Table full.
base_nas[err]:  [ev_log_t_SAI_ROUTE:Switch Id: 0], Route Add failed

//========

• The table usage calculation is as follows:

Example:

ip6 routes – 12 (show ipv6 route summary including all VRF)
ipv4 routes - 12 (show ip route summary including all VRF)
total routes = 36 = ip6-routes*2 + ip4-routes  <------ Note: IPv6 takes two entries.

l3 info shows the number of used/total-entries hosts/route entries.

//=======
OS10#system "sudo hshell -c 'l3 info'"
 hosts            :     12 / 147456
routes           :     31 /  32768
nexthops         :      9 /  65536
ecmp_groups      :      0 /   4096
overlay_ecmp_defrag  :      0
underlay_ecmp_defrag :      0
//=======

• Root Cause:
  • This issue could happen on any OS10 version.
  • The issue is due to the Broadcom hardware table being full, preventing new routes from being added.
• Supporting Documents:
  • Check the latest Dell SmartFabric OS10 User Guide for detailed command usage and hardware limitations.
  • Hardware limits for the S5200 platform:

• There is no intention to fix this issue in the OS10 code because it is a hardware limitation. Work with some of these options below to work around this limitation

• • Check the current hardware profile with the command below and adequate the routes and hosts within the limits. 

Note: The command was taken from s5248f.
OS10# show hardware forwarding-table mode all

 Mode             default scaled-l2  scaled-l3-routes scaled-l3-hosts
 L2 MAC Entries   163840   294912     32768            98304
 L3 Host Entries  147456   16384      16384            212992
 L3 Route Entries 32768    32768      131072           98304

• • Use different configurations to suppress unnecessaries routes or host entries installed in the switch.