Bug ID 1153853: Revision of default value for provision.restjavad.extramb to avoid OOM errors in restjavad

Last Modified: Jun 28, 2024

Affected Product(s):
BIG-IP TMOS(all modules)

Known Affected Versions:
14.1.5.1, 14.1.5.2, 14.1.5.3, 14.1.5.4, 14.1.5.6, 15.1.5, 15.1.5.1, 15.1.6, 15.1.6.1, 15.1.7, 15.1.8, 15.1.8.1, 15.1.8.2, 16.1.3.1, 16.1.3.2, 16.1.3.3, 16.1.3.4, 16.1.3.5, 17.0.0.1, 17.0.0.2

Fixed In:
17.1.0, 16.1.4, 15.1.9

Opened: Sep 12, 2022

Severity: 3-Major

Symptoms

- restjavad may be out memory as determined from restjavad logs, /var/log/restjavad.X.log, showing 'java.lang.OutOfMemoryError'. This may lead to frequent restjavad restarts and high CPU. - restjavad may instead, or as well, run many full garbage collection cycles one after another, causing high CPU. This will be shown by frequent logs with [FullGC] in /var/log/restjavad-gc.log.X.current

Impact

May have problems in TMUI with certain pages or tabs, such as network map with very config or SSLO or iLX related tabs. Other services that use REST API, internal and external to BIG-IP, may be impacted with low performance or service instability

Conditions

- Update to affected version: 14.1.5.1-, 15.1.7-15.1.8.2, 16.1.3.1-16.1.3.5, 17.0.0.1-17.0.0.2 or later versions. - Value of sys db restjavad.useextramb is true. - Value of sys db provision.restjavad.extramb is 192 or lower than previous restjavad heap size. - Use of REST API calls that need a lot of memory. Heavy users of REST API, such as SSL Orchestrator, may be very affected.

Workaround

Before upgrade - if you set sys db restjavad.useextramb to value false before install of new version you will have more restjavad memory, the default 384MB, after upgrade. tmsh modify sys db restjavad.useextramb value false If you restart restjavad you can see if that value works before upgrade. If you don't restart then it will come into effect after reboot. If that no longer has issues after update then leave that setting at false. Otherwise set back to true (no restart) and increase provision.restjavad.extramb as in After upgrade section below. After upgrade: Set sys db provision.restjavad.extramb to an appropriate value and restart restjavad. Run the following command: tmsh modify sys db provision.restjavad.extramb value X bigstart restart restjavad Iterate as necessary. The value of X is derived by using one of the following formulae: - When updating from versions before 14.1.4 and 15.1.3, to affected versions, a value that preserves the maximum previous restjavad heap size is: 192MB + 80% of MIN(provision.extramb|2500) the minimum possible heap size was: 192MB + 20% of MIN(provision.extramb|2500) The actual restjavad heap size would be between those extremes. SSLO systems would typically need the maximum. - When updating from 14.1.4-14.1.5, from 15.1.3-15.1.6.1 or from 16.0.x to affected versions: 384MB + 80% of MIN(provision.extramb|2500) - When updating from 16.1.0-16.1.3 or from 17.0.0.0 to affected versions: 384MB + 90% of MIN(provision.extramb|4000)

Fix Information

The fix here sets default restjavad size back to 384MB. The fix in ID 1153865 will set provision.restjavad.extramb to the maximum size that could have been in use before upgrade.

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips