Bug ID 499947: Improved performance loading thousands of Virtual Servers

Last Modified: Oct 16, 2023

Affected Product(s):
BIG-IP GTM(all modules)

Known Affected Versions:
11.5.1 HF1, 11.5.1 HF2, 11.5.1 HF3, 11.5.1 HF4, 11.5.1 HF5, 11.5.1 HF6, 11.5.1 HF7, 11.5.1 HF8, 11.5.1 HF9, 11.5.1 HF10, 11.5.1 HF11, 11.5.2 HF1, 11.5.3 HF1, 11.5.3 HF2, 11.5.4 HF1, 11.5.4 HF2, 11.5.4 HF3, 11.5.4 HF4, 11.4.1, 11.5.0, 11.5.1, 11.5.2, 11.6.0, 11.6.1, 11.6.2, 11.6.3, 11.6.3.1, 11.6.3.2, 11.6.3.3, 11.6.3.4, 11.6.4, 11.6.5, 11.6.5.1, 11.6.5.2, 11.6.5.3, 12.1.0 HF1, 12.1.0 HF2, 12.1.1 HF1, 12.1.1 HF2, 12.1.2 HF1, 12.1.2 HF2

Fixed In:
12.0.0, 11.6.0 HF5, 11.5.3

Opened: Jan 07, 2015

Severity: 2-Critical

Related Article: K16617

Symptoms

In v11.5.1 and newer, when loading thousands of Virtual Servers, mcpd might become overloaded, causing loads to take a long time, or fail entirely when mcpd times out and is restarted. This might be more severe if GTM was enabled.

Impact

Might cause long load times or configuration load failure because of mcpd timeout and restart.

Conditions

Thousands of Virtual Servers, GTM enabled. The problem is caused when tracking the state of Virtual Address changes and broadcasting those state changes under certain circumstances.

Workaround

Disable GTM. Reduce the number of Virtual Addresses.

Fix Information

The Virtual Address state change code was improved in multiple areas: 1. GTM is checked for provisioning. 2. Each individual Virtual Address is checked for GTM association before assuming it needs to be broadcast. 3. Virtual Address changes caused by the Virtual Server, Pool, or Virtual Address changes are processed at a higher priority. 4. Virtual Address changes caused by a GTM state change are processed after the Virtual Server changes in #3. 5. All Virtual Address changes are processed on a queue that limits the number per mcpd event loop, preventing Virtual Address status changes from blocking normal mcpd operations.

Behavior Change

The Virtual Address state change code was improved in multiple areas: 1. GTM is checked for provisioning. 2. Each individual Virtual Address is checked for GTM association before assuming it needs to be broadcast. 3. Virtual Address changes caused by the Virtual Server, Pool, or Virtual Address changes are processed at a higher priority. 4. Virtual Address changes caused by a GTM state change are processed after the Virtual Server changes in #3. 5. All Virtual Address changes are processed on a queue that limits the number per mcpd event loop, preventing Virtual Address status changes from blocking normal mcpd operations.

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips