Bug ID 519372: vCMP guest memory growth due to large number of /var/run/tmstats-rsync.* files.

Last Modified: Sep 13, 2023

Affected Product(s):
BIG-IP vCMP(all modules)

Known Affected Versions:
11.5.1 HF1, 11.5.1 HF2, 11.5.1 HF3, 11.5.1 HF4, 11.5.1 HF5, 11.5.1 HF6, 11.5.1 HF7, 11.5.1 HF8, 11.5.1 HF9, 11.5.1 HF10, 11.5.1 HF11, 11.5.2 HF1, 11.5.3 HF1, 11.5.3 HF2, 11.5.4 HF1, 11.5.4 HF2, 11.5.4 HF3, 11.5.4 HF4

Fixed In:
11.6.0 HF6

Opened: Apr 21, 2015

Severity: 3-Major

Related Article: K55273152

Symptoms

Extremely large and increasing number of files present, of the form /var/run/tmstats-rsync.*. This is a memory-backed directory, and these files are never automatically moved or deleted, hence the vCMP guest may eventually experience swap and out of memory conditions.

Impact

In swap and low memory conditions, the vCMP guest suffers performance problems and instability.

Conditions

vCMP guests upload statistics to the VCMP host periodically. In a small percentage of vCMP guests which have large configurations, these statistics take up an unusually high amount of space. This is not an error, but it exceeds the 6 MB limit that the host accepts. The host's refusal to accept the file triggers behavior in the guest that logs the condition to /var/run/tmstats-rsync.*. If the file size never decreases, this happens repeatedly and indefinitely.

Workaround

To work around this issue, you can disable guest health statistic collection on the vCMP host. To do so, perform one of the following procedures: Disabling statistic collection for the tmsh show vcmp health command. Impact of workaround: This procedure affects values returned by the tmsh show vcmp health stats command. 1. Log in to the command line of the vCMP host. If the device is a VIPRION, ensure you are logged in to the primary blade. 2. To disable statistic collection, type the following command: tmsh modify vcmp guest all capabilities add { stats-isolated-mode }.

Fix Information

The /var/run/tmstats-rsync.* files are no longer generated. Instead, statistics are kept in the vCMP guest to track failures to send stats to the host. You can see these by running the following command in the guest: tmctl -d blade vcmpd/rsync_stat. If the guest is a multi-slot guest on a VIPRION platform, this command shows separate stats for each slot it's run on.

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips