Bug ID 687472: APM Local DB database binary replication logs can fill up disk, leading to service outages

Last Modified: Aug 28, 2024

Affected Product(s):
BIG-IP APM(all modules)

Known Affected Versions:
13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1

Fixed In:
14.0.0, 13.1.3.2

Opened: Oct 06, 2017

Severity: 3-Major

Symptoms

BIG-IP APM uses mysql to maintain and synchronize local user objects. This synchronization creates binary log files that contain information that can be helpful for recovery in the face of database corruption. These files are not purged correctly by APM. They can fill up the disk, leading to unexpected behavior and service outages.

Impact

Potentially, a full disk can cause mysql to misbehave, leading to the logging process misbehaving, which can lead to BIG-IP downtime and service outages.

Conditions

APM is used with Local User Database in a High Availability (HA) environment.

Workaround

The files should be purged using the PURGE BINARY LOGS command in mysql. An example of this command: To purge logs (mysql binary log transaction files: /var/lib/mysql/mysqld-bin.xxx) that were generated before "2017-09-28 03:40:00" /usr/bin/mysql --user=root --password=`perl -MPassCrypt -nle 'print PassCrypt::decrypt_password($_)' /var/db/mysqlpw` -e 'use mysql; PURGE BINARY LOGS BEFORE "2017-09-28 03:40:00";' Administrators may also want to consider using MySQL system variable "expire_logs_days" which dictates the number of days for automatic binary log file removal. An example of this configuration: 1. edit /var/lib/mysql/my.cnf and add below variable: # only keep 7 days of binary log files (choose appropriately) expire_logs_days = 7 2. Restart MySQL (should be performed in a maintenance window): bigstart restart mysql Please note, administrators need to manually configure this on all high availability (HA) units and re-configure after an upgrade or EngHF installation.

Fix Information

None

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips