eSONIC: Elevated CPU Utilization For Syncd Container on Z9664
Summary: Dell PowerSwitch Z9664 switches may generate syslog messages indicating increased CPU usage by "syncd" in Enterprise SONiC Operating System [OS] 4.2.x and 4.4.x.
Symptoms
Identified Symptoms
Dell PowerSwitch Z9664 switches may generate syslog messages indicating increased CPU usage by "syncd."
Applicable Versions: Enterprise SONiC OS 4.2.x Enterprise SONiC OS 4.4.x Applicable Platform: PowerSwitch Z9664F-ON
PowerSwitch Z9664F-ON with RDMA over Converged Ethernet (RoCE) enabled and a large number of breakout ports could report elevated utilization of the ``` syncd ``` docker.
WARNING system#monitor: Current CPU usage of process syncd[5364] is 111.2% WARNING system#monitor: Current CPU usage of process syncd[5364] is 110.1% WARNING system#monitor: Current CPU usage of process syncd[5364] is 109.9%
Cause
Root Cause of Elevated CPU Utilization
Investigation revealed that the warning was generated without accounting for the load average on the system.
This behavior is related to the system's configuration and the enabled features, specifically, RDMA over Converged Ethernet (RoCE) and a large number of breakout ports on the PowerSwitch Z9664F-ON.
The warning messages are generated due to the high CPU usage of the "syncd" process, as shown in the following examples:
WARNING system#monitor: Current CPU usage of process syncd[5364] is 111.2% WARNING system#monitor: Current CPU usage of process syncd[5364] is 110.1% WARNING system#monitor: Current CPU usage of process syncd[5364] is 109.9%
Resolution
Resolution for Elevated CPU Utilization
In Enterprise SONiC OS release 4.5.0 and newer, the load average is factored into generating the elevated syncd utilization warning in the syslog.
To address the issue, the following steps can be taken:
- Monitor the system for any updates to the Enterprise SONiC OS.
- Upgrade to the latest version of Enterprise SONiC OS when available.
It is essential to note that this behavior does not impact traffic.