Bug ID 559023: Socket Affinity, CPU and Memory Reservations and NUMA architectures can cause daemon cores from timed out operations

Last Modified: Sep 13, 2023

Affected Product(s):
BIG-IP All(all modules)

Known Affected Versions:
13.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 13.1.1.5, 13.1.3, 13.1.3.1, 13.1.3.2, 13.1.3.3, 13.1.3.4, 13.1.3.5, 13.1.3.6, 13.1.4, 13.1.4.1, 13.1.5, 13.1.5.1

Opened: Nov 19, 2015

Severity: 2-Critical

Symptoms

bd, tmm or mcpd has crashed several instances of Clock Advanced Notices are seen in the ltm log

Impact

The VE is not receiving adequate access to hypervisor resources for proper software operation

Conditions

The hypervisor must have two or more sockets

Workaround

Adjust the CPU, Memory and Scheduling affinity controls to grant low latency access to discrete Hypervisor components On VMware, there is a resources tab or page. 1) CPU Reservation Insure the CPU reservation is appropriate for the MHz required. For example, if the hypervisor has 2.0GHz cores and the VE is set to 4 cores, it will need 4x2.0GHz reserved for 8GHz, or 8000MHz. 2) Memory Reservation The memory reservation should match the amount of memory the VE is defined with. An 8192MB VE with 4 cores will have 8192MB of memory reserved. Environments using shares should adjust share allocations such that the VE will always receive 100% of its memory at any given time. If priority is used, always grant the VE memory at high priority. 3) CPU Affinity Under Advanced CPU there is a control for Scheduling Affinity that should be adjusted to put the VE on the CPUs of just one physical socket. On a 12 core/24 thread machine, this will be set as 0-5 or 6-11. For even lower latency assign the BIG-IP VE 1:1 with the hypervisor cores and assign no other Virtual Machine to these cores - but this is not usually required. 4) NUMA affinity Under Advanced Memory, adjust the memory from nodes preference to match 0 or 1 for the first or second set of the CPU affinity mask.

Fix Information

None

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips