Bug ID 1035485: When BIG-IQ is configured with data collection devices with 32 GB RAM and runs out memory and SWAP every 8 to 10 days

Last Modified: Dec 07, 2023

Affected Product(s):
BIG-IQ AppIQ(all modules)

Known Affected Versions:
7.0.0,,, 7.1.0,,,, 7.1.6,, 7.1.7,,, 7.1.8,,,,,, 7.1.9,,,, 8.0.0,, 8.1.0,,

Opened: Jul 22, 2021

Severity: 3-Major

Related Article: https://cdn.f5.com/product/bugtracker/ID1035485.html


When BIG-IQ is configured with data collection devices (DCDs) and the DCDs have only 32 GB of RAM on each, the DCD logs the following message: [root@bigiq1:Active:Standalone] config # cat /var/log/appiq/agentmanager.log | grep -i 'rate:' | grep Elastic INFO c.f.a.i.e.c.p.ElasticSearchPublisher [out-publish-pipeline-23] Elasticsearch publish rate: [15:08; docs/sec: 4226.00, 4316.44, 4439.92]


The DCD eventually runs out of resources and stops some internal processes to free up resources.


When the DCD has only 32 GB of RAM and the docs write rate for each DCD exceeds ~2k/sec.


Option 1) Configure the DCD with sufficient RAM using the following table as guidance ES: Elasticsearch ES docs/sec (on each DCD) RAM ---------------------------------- 0 - 2,000 32 GB 2,000 - 5000 64 GB 5000 - X A single DCD can not handle more than 5k docs/sec (cpu limit, not related to memory limit) and you will need more DCDs Option 2) (less recommended): If you cannot get a DCD with 64 GB, increase the swap memory from 1 GB to 4 GB or more. For example, to increase the swap memory to 16 GB. NOTE: If hard disks (HDD) are used instead of SDD, replace sda with hda. --- swapon -s # record the filename location to be used with the blkid command, below Sample Output: Filename Type Size Used Priority /dev/dm-3 partition 1048572 0 -1 swapoff -a # this removes all current swap provisioning blkid /dev/dm-3 # record UUID file -s -L /dev/dm-3 # this confirms that the location referenced is valid) lvresize --size 16G /dev/mapper/vg--db--sda-dat.swapvol.1 mkswap -U <UUID from blkid or file command> /dev/mapper/vg--db--sda-dat.swapvol.1 swapon /dev/dm-3 swapon -s Note: Output of this command should resemble what was displayed earlier, except the size should be 16777212. reboot swapon -s ---

Fix Information


Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips