Bug ID 1015005: Elastic Search is not properly functioning in a very high latency network environment

Last Modified: Jul 12, 2023

Affected Product(s):
BIG-IQ AppIQ(all modules)

Known Affected Versions:
7.0.0, 7.0.0.1, 7.0.0.2, 7.1.0, 7.1.0.1, 7.1.0.2, 7.1.0.3, 7.1.6, 7.1.6.1, 7.1.7, 7.1.7.1, 7.1.7.2, 7.1.8, 7.1.8.1, 7.1.8.2, 7.1.8.3, 7.1.8.4, 7.1.8.5, 7.1.9, 7.1.9.7, 7.1.9.8, 7.1.9.9, 8.0.0, 8.0.0.1

Opened: Apr 28, 2021

Severity: 3-Major

Symptoms

Due to latency issues (about 125 ms or higher)to a DCD or CM, unassigned shards can cause performance issues with Elasticsearch.

Impact

The ES indexes can become red or yellow as it accumulates unassigned shards. As a result, this can lead to data loss.

Conditions

The network environment has high latency between Elasticsearch (ES) nodes.

Workaround

1.For each ES node we need to edit the file: /var/config/rest/elasticsearch/config/elasticsearch.yml and add the following (and run "bigstart restart elasticsearch" after the edit): discovery: zen: fd: ping_interval: 15s ping_timeout: 60s ping_retries: 5 2.On each ES index (except for .opendistro-security) curl -XPUT --insecure https://localhost:9200/<index name>/_settings?pretty=true -H 'Content-Type: application/json' -d '{ "index": { "allocation": { "max_retries": 15 } } }' (Instead of each index name you can use also '*', for example statistics*)

Fix Information

None

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips