Bug ID 499348: System statistics may fail to update, or report negative deltas due to delayed stats merging

Last Modified: Oct 29, 2019

Bug Tracker

Affected Product:  See more info
BIG-IP All(all modules)

Known Affected Versions:
11.5.0, 11.5.1, 11.5.1 HF1, 11.5.1 HF10, 11.5.1 HF11, 11.5.1 HF2, 11.5.1 HF3, 11.5.1 HF4, 11.5.1 HF5, 11.5.1 HF6, 11.5.1 HF7, 11.5.1 HF8, 11.5.1 HF9, 11.5.2, 11.5.2 HF1, 11.5.3, 11.5.3 HF1, 11.5.3 HF2, 11.5.4, 11.5.4 HF1, 11.5.4 HF2, 11.5.4 HF3, 11.5.4 HF4, 11.5.5, 11.5.6, 11.5.7, 11.5.8, 11.5.9, 11.6.0, 11.6.0 HF1, 11.6.0 HF2, 11.6.0 HF3, 11.6.0 HF4, 11.6.0 HF5, 11.6.0 HF6, 11.6.0 HF7, 11.6.0 HF8, 11.6.1, 11.6.1 HF1, 11.6.1 HF2, 11.6.2, 11.6.2 HF1, 11.6.3, 11.6.3.1, 11.6.3.2, 11.6.3.3, 11.6.3.4, 11.6.4, 11.6.5, 12.0.0, 12.0.0 HF1, 12.0.0 HF2, 12.0.0 HF3, 12.0.0 HF4, 12.1.0, 12.1.0 HF1, 12.1.0 HF2, 12.1.1, 12.1.1 HF1, 12.1.1 HF2, 12.1.2, 12.1.2 HF1, 12.1.2 HF2, 12.1.3, 12.1.3.1, 12.1.3.2, 12.1.3.3, 12.1.3.4, 12.1.3.5, 12.1.3.6, 12.1.3.7, 12.1.4, 12.1.4.1, 12.1.5, 13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 13.1.1.5, 13.1.3, 13.1.3.1, 14.0.0, 14.0.0.1, 14.0.0.2, 14.0.0.3, 14.0.0.4, 14.0.0.5, 14.0.1, 14.0.1.1, 14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.4, 14.1.0.5, 14.1.0.6, 14.1.2, 14.1.2.1, 15.0.0, 15.0.1

Opened: Jan 05, 2015
Severity: 3-Major
Related AskF5 Article:
K21811305

Symptoms

Under some conditions, the BIG-IP system might fail to report statistics over time. This can manifest as statistics reporting unchanging statistics (e.g., all zeroes (0)), or as sudden spikes in traffic, or as negative deltas in some counters. The system performance graphs will also appear to have gaps / be missing data at the times that this occurs.

Impact

Statistics fail to merge, which results in incorrect view of system behavior and operation.

Conditions

This occurs when there are frequent changes occurring to the underlying statistics data structures. This might occur under the following conditions: -- The system is spawning/reaping processes on a frequent basis (e.g., when there is a large number of external monitors). -- iRules are frequently using 'SSL::profile' to select different SSL profiles on a virtual server (this can cause per-virtual server, per-profile statistics to be created and deleted on a regular basis).

Workaround

This issue has two workarounds: 1. Reduce the frequency of changes in the statistics data structures. The specific action to take depends on what is triggering them. To do so, use any or all of the following: -- Reduce the frequency of configuration changes. -- Reduce the use of 'SSL::profile' in iRules. -- Reduce the number/frequency of processes being spawned by the system. 2. Switch statistics roll-ups to the 'slow_merge' method, which causes the system to spend more CPU merging statistics. To do so, set the 'merged.method' DB key to 'slow_merge' using the following command: tmsh modify sys db merged.method value slow_merge.

Fix Information

None

Behavior Change