Symptoms

When BIG-IQ is configured with data collection devices (DCDs) and the DCDs have only 32 GB of RAM on each, the DCD logs the following message: [root@bigiq1:Active:Standalone] config # cat /var/log/appiq/agentmanager.log | grep -i 'rate:' | grep Elastic INFO c.f.a.i.e.c.p.ElasticSearchPublisher [out-publish-pipeline-23] Elasticsearch publish rate: [15:08; docs/sec: 4226.00, 4316.44, 4439.92]

Impact

The DCD eventually runs out of resources and stops some internal processes to free up resources.

Conditions

When the DCD has only 32 GB of RAM and the docs write rate for each DCD exceeds ~2k/sec.

Workaround

Option 1) Configure the DCD with sufficient RAM using the following table as guidance ES: Elasticsearch ES docs/sec (on each DCD) RAM ---------------------------------- 0 - 2,000 32 GB 2,000 - 5000 64 GB 5000 - X A single DCD can not handle more than 5k docs/sec (cpu limit, not related to memory limit) and you will need more DCDs Option 2) (less recommended): If you cannot get a DCD with 64 GB, increase the swap memory from 1 GB to 4 GB or more. For example, to increase the swap memory to 16 GB. NOTE: If hard disks (HDD) are used instead of SDD, replace sda with hda. --- swapon -s # record the filename location to be used with the blkid command, below Sample Output: Filename Type Size Used Priority /dev/dm-3 partition 1048572 0 -1 swapoff -a # this removes all current swap provisioning blkid /dev/dm-3 # record UUID file -s -L /dev/dm-3 # this confirms that the location referenced is valid) lvresize --size 16G /dev/mapper/vg--db--sda-dat.swapvol.1 mkswap -U <UUID from blkid or file command> /dev/mapper/vg--db--sda-dat.swapvol.1 swapon /dev/dm-3 swapon -s Note: Output of this command should resemble what was displayed earlier, except the size should be 16777212. reboot swapon -s ---

Fix Information

None

Behavior Change