Bug ID 947341: MySQL generates multiple error 24 (too many files open) for PRX.REQUEST_LOG DB tables files

Last Modified: Jul 23, 2021

Bug Tracker

Affected Product:  See more info
BIG-IP ASM(all modules)

Known Affected Versions:
14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.5, 14.1.0.6, 14.1.2, 14.1.2.1, 14.1.2.2, 14.1.2.3, 14.1.2.4, 14.1.2.5, 14.1.2.6, 14.1.2.7, 14.1.2.8, 14.1.3, 14.1.3.1, 14.1.4, 15.0.0, 15.0.1, 15.0.1.1, 15.0.1.2, 15.0.1.3, 15.0.1.4, 15.1.0, 15.1.0.1, 15.1.0.2, 15.1.0.3, 15.1.0.4, 15.1.0.5, 15.1.1, 15.1.2, 15.1.2.1, 16.0.0, 16.0.0.1, 16.0.1, 16.0.1.1, 16.1.0

Fixed In:
16.0.1.2, 15.1.3, 14.1.4.1

Opened: Sep 22, 2020
Severity: 3-Major

Symptoms

1) var/lib/mysql/mysqld.err is filled with log entries containing: ------------ 200824 11:04:43 [ERROR] mysqld: Can't open file: './PRX/REQUEST_LOG.frm' (errno: 24) 200824 11:18:46 [ERROR] mysqld: Can't open file: './PRX/REQUEST_LOG.frm' (errno: 24) 200824 11:35:58 [ERROR] mysqld: Can't open file: './PRX/REQUEST_LOG.frm' (errno: 24) ------------ 2) There are a lot of PRX.REQUEST_LOG partitions, in some cases up to 1024, many of which are empty.

Impact

MySQL out of resources when opening files PRX.REQUEST_LOG Corrupt

Conditions

ASM/AVR provisioned

Workaround

0) If the /appdata partition is filled to 100% and MySQL restarts continuously, refer to the following Knowledge Articles: https://support.f5.com/csp/article/K14956 https://support.f5.com/csp/article/K42497314 1) To identify the empty partitions, look into: SELECT * FROM INFORMATION_SCHEMA.PARTITIONS WHERE table_name = 'REQUEST_LOG' AND table_schema = 'PRX'\G 2) For every partition that is empty, manually (or via shell script) execute this sql: ALTER TABLE PRX.REQUEST_LOG DROP PARTITION empty_partition_name where 'empty_partition_name' is the partition name as 'p100001' 4) Increase 'open_files_limit' to '10000'. -------------------------------- In the /etc/my.cnf file: 1. Change the value of the 'open_files_limit' parameter to 10000. 2. Restart MySQL: bigstart restart mysql -------------------------------- 5) pkill asmlogd Note: This workaround does not survive upgrade. It must be reapplied after every upgrade until the upgraded version contains a fix.

Fix Information

This release increases 'open_files_limit' to '10000'.

Behavior Change