Bug ID 519372: vCMP guest memory growth due to large number of /var/run/tmstats-rsync.* files.

Last Modified: Nov 07, 2022

Bug Tracker

Affected Product:  See more info
BIG-IP vCMP(all modules)

Known Affected Versions:
11.6.0, 11.6.0 HF1, 11.6.0 HF2, 11.6.0 HF3, 11.6.0 HF4, 11.6.0 HF5

Fixed In:
11.6.0 HF6

Opened: Apr 21, 2015
Severity: 3-Major
Related Article:
K55273152

Symptoms

Extremely large and increasing number of files present, of the form /var/run/tmstats-rsync.*. This is a memory-backed directory, and these files are never automatically moved or deleted, hence the vCMP guest may eventually experience swap and out of memory conditions.

Impact

In swap and low memory conditions, the vCMP guest suffers performance problems and instability.

Conditions

vCMP guests upload statistics to the VCMP host periodically. In a small percentage of vCMP guests which have large configurations, these statistics take up an unusually high amount of space. This is not an error, but it exceeds the 6 MB limit that the host accepts. The host's refusal to accept the file triggers behavior in the guest that logs the condition to /var/run/tmstats-rsync.*. If the file size never decreases, this happens repeatedly and indefinitely.

Workaround

To work around this issue, you can disable guest health statistic collection on the vCMP host. To do so, perform one of the following procedures: Disabling statistic collection for the tmsh show vcmp health command. Impact of workaround: This procedure affects values returned by the tmsh show vcmp health stats command. 1. Log in to the command line of the vCMP host. If the device is a VIPRION, ensure you are logged in to the primary blade. 2. To disable statistic collection, type the following command: tmsh modify vcmp guest all capabilities add { stats-isolated-mode }.

Fix Information

The /var/run/tmstats-rsync.* files are no longer generated. Instead, statistics are kept in the vCMP guest to track failures to send stats to the host. You can see these by running the following command in the guest: tmctl -d blade vcmpd/rsync_stat. If the guest is a multi-slot guest on a VIPRION platform, this command shows separate stats for each slot it's run on.

Behavior Change