Bug ID 756311: High CPU during erroneous deletion

Last Modified: Nov 04, 2019

Bug Tracker

Affected Product:  See more info
BIG-IP PEM(all modules)

Known Affected Versions:
12.1.0, 12.1.0 HF1, 12.1.0 HF2, 12.1.1, 12.1.1 HF1, 12.1.1 HF2, 12.1.2, 12.1.2 HF1, 12.1.2 HF2, 12.1.3, 12.1.3.1, 12.1.3.2, 12.1.3.3, 12.1.3.4, 12.1.3.5, 12.1.3.6, 12.1.3.7, 12.1.4, 12.1.4.1, 12.1.5, 13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 13.1.1.5, 14.0.0, 14.0.0.1, 14.0.0.2, 14.0.0.3, 14.0.0.4, 14.0.0.5, 14.0.1, 14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.4, 14.1.0.5, 14.1.0.6, 14.1.2

Fixed In:
15.0.0, 14.1.2.1, 14.0.1.1, 13.1.3

Opened: Jan 23, 2019
Severity: 3-Major

Symptoms

The utilization of some CPU cores increases and remains high for a long time. Rebooting just one blade can cause the high CPU usage to move to another blade in the chassis. There might be messages similar to the following in tmm logs: -- notice PEM: spm_subs_id_consistency_check_cb: Session 10.1.10.10%0-4a89723e; Instance ID mismatch ERR_OK for subscriber id 310012348494 with 10.1.10.10-0-4a8987ea. -- notice PEM: spm_subs_id_consistency_check_cb: Session 10.1.18.10%0-4a8b850c; Look up returned err ERR_OK for subscriber id 3101512411557

Impact

The CPU usage is coming from an erroneous cleanup function, which is only running on a TMM when it's not busy; traffic is not expected to have a significant impact. However, recovering may result in a cluster-wide TMM restart, if the CPU usage does not subside. Traffic disrupted while tmm restarts.

Conditions

The exact conditions under which this occurs are not fully understood, but one way it can be triggered is when a single TMM is crashing on a chassis system.

Workaround

Delete all subscribers from the CLI.

Fix Information

None

Behavior Change