Bug ID 944785: Admd restarting constantly. Out of memory due to loading malformed state file

Last Modified: May 29, 2024

Affected Product(s):
BIG-IP ASM(all modules)

Known Affected Versions:
14.0.0, 14.0.0.1, 14.0.0.2, 14.0.0.3, 14.0.0.4, 14.0.0.5, 14.0.1, 14.0.1.1, 14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.5, 14.1.0.6, 14.1.2, 14.1.2.1, 14.1.2.2, 14.1.2.3, 14.1.2.4, 14.1.2.5, 14.1.2.6, 14.1.2.7, 14.1.2.8, 14.1.3, 15.0.0, 15.0.1, 15.0.1.1, 15.0.1.2, 15.0.1.3, 15.0.1.4, 15.1.0, 15.1.0.1, 15.1.0.2, 15.1.0.3, 15.1.0.4, 15.1.0.5, 15.1.1, 16.0.0, 16.0.0.1, 16.0.1, 16.0.1.1

Fixed In:
16.1.0, 16.0.1.2, 15.1.2, 14.1.3.1

Opened: Sep 14, 2020

Severity: 3-Major

Symptoms

Admd consumes more than 10GB of RSS Wrong signature statistics and possible memory corruption, potentially results in high memory consumption.

Impact

ADMD not working, ADMD constantly restarting, consuming all of the system memory. Out of memory. ADMD killed due to memory consumption

Conditions

-- Upgrading from 13.x, 14.x to 15.x -- Device service clustering configuration -- App-Protect-DOS signatures configured.

Workaround

Make sure that all the devices within a cluster are running compatible state file version (either all with versions before 15.1.0.x or after), if not, then: 1. Stop ADMD on all of those devices: bigstart stop admd 2. Upgrade or Downgrade the BIG-IP version to match the above criteria. 3. Remove the old state files: rm -rf /var/run/adm/* 4. Start ADMD: bigstart start ADMD If there is an issue on a single blade device, then: 1. Stop ADMD on all of those machines: bigstart stop admd 2. Remove the old state files: rm -rf /var/run/adm/* 3. Start ADMD: bigstart start ADMD

Fix Information

No more memory corruption, no OOM nor ADMD restarts.

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips