Bug ID 959957: Asmlogd stops deleting old protobufs

Last Modified: Dec 20, 2023

Affected Product(s):
BIG-IP ASM(all modules)

Known Affected Versions:
11.6.5, 11.6.5.1, 11.6.5.2, 11.6.5.3, 12.0.0, 12.0.0 HF1, 12.1.0 HF1, 12.0.0 HF2, 12.1.0 HF2, 12.0.0 HF3, 12.0.0 HF4, 12.1.1 HF1, 12.1.1 HF2, 12.1.2 HF1, 12.1.2 HF2, 12.1.0, 12.1.1, 12.1.2, 12.1.3, 12.1.3.1, 12.1.3.2, 12.1.3.3, 12.1.3.4, 12.1.3.5, 12.1.3.6, 12.1.3.7, 12.1.4, 12.1.4.1, 12.1.5, 12.1.5.1, 12.1.5.2, 12.1.5.3, 12.1.6, 13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 13.1.1.5, 13.1.3, 13.1.3.1, 13.1.3.2, 13.1.3.3, 13.1.3.4, 13.1.3.5, 13.1.3.6, 13.1.4, 13.1.4.1, 13.1.5, 13.1.5.1, 14.0.0, 14.0.0.1, 14.0.0.2, 14.0.0.3, 14.0.0.4, 14.0.0.5, 14.0.1, 14.0.1.1, 14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.5, 14.1.0.6, 14.1.2, 14.1.2.1, 14.1.2.2, 14.1.2.3, 14.1.2.4, 14.1.2.5, 14.1.2.6, 14.1.2.7, 14.1.2.8, 14.1.3, 14.1.3.1, 14.1.4, 14.1.4.1, 14.1.4.2, 14.1.4.3, 14.1.4.4, 14.1.4.5, 14.1.4.6, 14.1.5, 14.1.5.1, 14.1.5.2, 14.1.5.3, 14.1.5.4, 14.1.5.6, 15.0.0, 15.0.1, 15.0.1.1, 15.0.1.2, 15.0.1.3, 15.0.1.4, 15.1.0, 15.1.0.1, 15.1.0.2, 15.1.0.3, 15.1.0.4, 15.1.0.5, 15.1.1, 15.1.2, 15.1.2.1, 15.1.3, 15.1.3.1, 15.1.4, 15.1.4.1, 15.1.5, 15.1.5.1, 15.1.6, 15.1.6.1, 15.1.7, 15.1.8, 15.1.8.1, 15.1.8.2, 15.1.9, 15.1.9.1, 15.1.10, 15.1.10.2, 15.1.10.3, 16.0.0, 16.0.0.1, 16.0.1, 16.0.1.1, 16.0.1.2, 16.1.0, 16.1.1, 16.1.2, 16.1.2.1, 16.1.2.2, 16.1.3, 16.1.3.1, 16.1.3.2, 16.1.3.3, 16.1.3.4, 16.1.3.5, 16.1.4, 16.1.4.1, 16.1.4.2

Opened: Oct 28, 2020

Severity: 3-Major

Symptoms

Asmlogd restarts and there are asmlogd errors: asmlogd|ERR|Oct 19 13:46:35.199|6005|,,asmlogd ended unexpectedly asmlogd|ERR|Oct 19 13:46:35.203|6005|,,Can't call method "size" on an undefined value at /usr/local/share/perl5/F5/RequestLog.pm line 1902.

Impact

Old protobuf files are not cleaned up.

Conditions

Disk is full -- the following warnings are being displayed: err diskmonitor[3483]: 011d0004:3: Disk partition /var/asmdata1 (slot 1) has only 0% free

Workaround

0) If the /appdata partition is filled to 100% and MySQL restarts continuously, refer to the following Knowledge Articles: https://support.f5.com/csp/article/K14956 https://support.f5.com/csp/article/K42497314 1) To identify the empty partitions, look into: SELECT * FROM INFORMATION_SCHEMA.PARTITIONS WHERE table_name = 'REQUEST_LOG' AND table_schema = 'PRX'\G 2) For every partition that is empty, manually (or via shell script) execute this sql: ALTER TABLE PRX.REQUEST_LOG DROP PARTITION empty_partition_name where 'empty_partition_name' is the partition name as 'p100001' 4) Increase 'open_files_limit' to '10000'. -------------------------------- In the /etc/my.cnf file: 1. Change the value of the 'open_files_limit' parameter to 10000. 2. Restart MySQL: bigstart restart mysql -------------------------------- 5) pkill asmlogd Note: This workaround does not survive upgrade. It must be reapplied after every upgrade until the upgraded version contains a fix.

Fix Information

None

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips