Bug ID 1002969: csyncd can consume excessive CPU time

Last Modified: Sep 29, 2022

Bug Tracker

Affected Product:  See more info
BIG-IP Install/Upgrade, TMOS(all modules)

Known Affected Versions:
13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 13.1.1.5, 13.1.3, 13.1.3.1, 13.1.3.2, 13.1.3.3, 13.1.3.4, 13.1.3.5, 13.1.3.6, 13.1.4, 13.1.4.1, 13.1.5, 13.1.5.1, 14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.5, 14.1.0.6, 14.1.2, 14.1.2.1, 14.1.2.2, 14.1.2.3, 14.1.2.4, 14.1.2.5, 14.1.2.6, 14.1.2.7, 14.1.2.8, 14.1.3, 14.1.3.1, 14.1.4, 14.1.4.1, 14.1.4.2, 14.1.4.3, 14.1.4.4, 14.1.4.5, 14.1.4.6, 14.1.5, 14.1.5.1, 14.1.5.2, 15.1.0, 15.1.0.1, 15.1.0.2, 15.1.0.3, 15.1.0.4, 15.1.0.5, 15.1.1, 15.1.2, 15.1.2.1, 15.1.3, 15.1.3.1, 15.1.4, 15.1.4.1, 15.1.5, 15.1.5.1, 15.1.6, 15.1.6.1, 15.1.7, 16.1.0, 16.1.1, 16.1.2, 16.1.2.1, 16.1.2.2, 16.1.3, 16.1.3.1, 16.1.3.2

Opened: Mar 16, 2021
Severity: 3-Major

Symptoms

Following a configuration change or software upgrade, the "csyncd" process becomes always busy, consuming excessive CPU.

Impact

The overuse of CPU resources by "csyncd" may starve other control-plane processes. Handling of payload network traffic by the data plane is not directly affected.

Conditions

-- occurs on a multiblade VIPRION chassis -- may occur with or without vCMP -- may occur after configuring F5 Telemetry Streaming, but may also occur in other circumstances -- large numbers of files are contained in one or more of the directories being sync'ed between blades

Workaround

To mitigate the processing load, identify which directory or directories contain excessive numbers of files being replicated between blades by "csyncd". If this replication is not absolutely needed, such a directory can be removed from the set of directories being sync'ed. For example: if there are too many files being generated in the "/run/pamcache" directory (same as "/var/run/pamcache"), remove this directory from the set being acted upon by "csyncd" by running the following commands to comment-out the associated lines in the configuration file: # clsh "cp /etc/csyncd.conf /etc/csyncd.conf.$(date +%Y%m%d_%H%M%S)" # clsh "sed -i '/run\/pamcache/,+2s/^/#/' /etc/csyncd.conf" # clsh "bigstart restart csyncd" If the problem was observed soon after the installation of F5 Telemetry Streaming, the configuration can be adjusted to make csyncd ignore the related files in a subdirectory of "/var/config/rest/iapps". Run the following commands: # clsh "cp /etc/csyncd.conf /etc/csyncd.conf.$(date +%Y%m%d_%H%M%S)" # clsh "sed -i '/\/var\/config\/rest\/iapps/a \ \ \ \ \ \ \ \ ignore f5-telemetry' /etc/csyncd.conf" # clsh "bigstart restart csyncd"

Fix Information

None

Behavior Change