Networking SONiC: System Memory Usage in SONiC - "buff/available"

Question 1: What is buff/cache?

"buff/cache" is the amount of memory used for file system buffers and cache. This memory usage is a crucial part of how Linux manages and optimizes system performance. Let us break down what "buff/cache" means:

Buffers:
Buffers in this context indicate a portion of system memory (RAM) that is used to temporarily hold data that is being read from or written to disk. When a program reads data from a file, for example, the data is first loaded into a buffer in memory. Buffers help improve disk I/O performance because reading or writing to RAM is faster than accessing the physical disk. Buffers can also ensure that the data is written to disk more efficiently, reducing the number of disk I/O operations.

Cache:
Cache, in this context, indicates the portion of memory used to store frequently accessed data from files and directories. Linux uses a mechanism called the "page cache" to cache data from files and directories that have been read or written to recently. This cached data is kept in memory so that if the same data is needed again, it can be quickly retrieved from RAM instead of reading it from the slower disk storage. This cache can significantly improve system performance by reducing the requirement to access the disk for frequently used data.

When applications or the operating system need more memory for active processes, the buffer/cache memory can quickly be released to accommodate these needs.

The "top" command displays this value. It also displays the system's current status, CPU usage, memory usage.

admin@sonic:~$ top
top - 16:43:51 up 8 days, 18:15,  1 user,  load average: 1.54, 1.50, 1.51
Tasks: 365 total,   2 running, 363 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.8 us,  4.2 sy,  0.0 ni, 93.1 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  15482.2 total,  11292.8 free,   2299.6 used,   1889.8 buff/cache 
MiB Swap:      0.0 total,      0.0 free,      0.0 used.  12965.1 avail Mem

Question 2: What is the threshold for high buffer/cache memory utilization?

Buffer/cache memory is used as a cache to store frequently accessed data. Hence it is possible that a significant portion of available RAM ends up allocated for this purpose. What is to be noted is that buffer/cache memory usage is highly dynamic and constantly changing based on the system demand. Therefore, It is more valuable to monitor the overall memory usage (or the corresponding "avail Mem") and ensure that the system has sufficient free memory for running applications and processes.

top - 14:17:13 up 207 days,  1:59,  1 user,  load average: 2.66, 2.19, 2.03
Tasks: 386 total,   2 running, 384 sleeping,   0 stopped,   0 zombie
%Cpu(s): 12.5 us,  9.7 sy,  0.0 ni, 77.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  15482.6 total,   8571.4 free,   2692.5 used,   4218.8 buff/cache
MiB Swap:      0.0 total,      0.0 free,      0.0 used.  12773.1 avail Mem   <--- system's total available RAM

A helpful command to express the percentage of system RAM available for use on a Dell Enterprise SONiC switch:

free | grep Mem | awk '{print $7/$2 * 100 }'

This command divides "avail Mem" by "MiB Mem total."

Here is another example comparing the above commands and indicating their relationships for your reference:

admin@Leaf3B:~$ top
top - 21:49:49 up  1:25,  1 user,  load average: 0.59, 0.60, 0.64
Tasks: 329 total,   1 running, 328 sleeping,   0 stopped,   0 zombie
%Cpu(s):  7.9 us,  4.2 sy,  0.0 ni, 87.5 id,  0.0 wa,  0.0 hi,  0.4 si,  0.0 st
KiB Mem : 15928852 total, 12074024 free,  2439884 used,  1414944 buff/cache
KiB Swap:        0 total,        0 free,        0 used. 13120520 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
21685 root      20   0  172992  33116   8436 S  52.9  0.2   3:16.11 python3.6
 1278 root      20   0   92872  15116   3264 S  17.6  0.1   2:24.49 redis-serv+
 5366 admin     20   0   43540   3812   3020 R  17.6  0.0   0:00.03 top
18469 root      20   0 1643080 451968  78276 S  11.8  2.8   8:17.32 syncd
 1097 root      20   0  136440  15188   5912 S   5.9  0.1   0:07.75 watchdogd
19856 admin     20   0  334880  61900  35296 S   5.9  0.4   0:00.44 docker
20985 root      20   0  159488  18616   5036 S   5.9  0.1   0:19.22 xcvrd
    1 root      20   0   57768   7820   5544 S   0.0  0.0   0:32.58 systemd
    2 root      20   0       0      0      0 S   0.0  0.0   0:00.00 kthreadd
    3 root      20   0       0      0      0 S   0.0  0.0   0:00.91 ksoftirqd/0
    4 root      20   0       0      0      0 S   0.0  0.0   0:01.90 kworker/0:0
    5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:+
    7 root      20   0       0      0      0 S   0.0  0.0   0:06.93 rcu_sched
    8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh
    9 root      rt   0       0      0      0 S   0.0  0.0   0:00.39 migration/0
   10 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 lru-add-dr+
   11 root      rt   0       0      0      0 S   0.0  0.0   0:00.00 watchdog/0

admin@Leaf3B:~$ free
              total        used        free      shared  buff/cache   available
Mem:       15928852     2438836    12074940       36060     1415076    13121528
Swap:             0           0           0
admin@Leaf3B:~$ free | grep Mem | awk '{print $7/$2 * 100 }'
82.3662
admin@Leaf3B:~$