Bug ID 873877: Kernel page allocation failure seen on VIPRION blades

Last Modified: May 29, 2024

Affected Product(s):
BIG-IP Install/Upgrade, TMOS(all modules)

Known Affected Versions:
14.1.2.3, 14.1.2.2, 14.1.2.1, 14.1.2, 14.1.0.6

Fixed In:
14.1.2.5

Opened: Jan 29, 2020

Severity: 3-Major

Symptoms

Despite having free memory, the BIG-IP system frequently logs kernel page allocation failures to the /var/log/kern.log file: warning kernel: : [7673174.106142] swapper/6: page allocation failure: order:2, mode:0x104020

Impact

As different processes can experience this issue, the system may behave unpredictably. For example, it is possible for installation to fail as a result of this issue. Other processes may not exhibit any side effect as a result of this issue. The exact impact depends on which process becomes affected and how this process is designed to handle such a failure to allocate memory.

Conditions

This issue is known to occur on the following VIPRION blade models: - B2250 (A112) - B4300 (A108) - B4340N (A110) - B4450 (A114)

Workaround

You can work around this issue by increasing the value of the min_free_kbytes kernel parameter. This controls the amount of memory that is kept free for use by special reserves. It is recommend to increase this as follows: -- 64 MB (65536 KB for B2250 blades). -- 48 MB (49152 KB for B4300 blades). -- 128 MB (131072 KB forB4450 blades). You must do this on each blade installed in the system. When instantiating this workaround, you must consider whether you want the workaround to survive only reboots, or to survive reboots, upgrades, RMAs, etc. This is an important consideration to make, as you should stop using this workaround when this issue is fixed in a future version of BIG-IP software. So consider the pros and cons of each approach before choosing one. -- If you want the workaround to survive reboots only, perform the following procedure: 1) Log on to the advanced shell (BASH) of the primary blade of the affected VIPRION system. 2) Run the following commands (with the desired amount in KB): # clsh "sysctl -w vm.min_free_kbytes=131072" # clsh "echo -e '\n# Workaround for ID753650' >> /etc/sysctl.conf" # clsh "echo 'vm.min_free_kbytes = 131072' >> /etc/sysctl.conf" -- If you want the workaround to survive reboots, upgrades, RMAs, etc., perform the following procedure: 1) Log on to the advanced shell (BASH) of the primary blade of the affected VIPRION system. 2) Run the following commands (with the desired amount in KB): # clsh "sysctl -w vm.min_free_kbytes=131072" # echo -e '\n# Workaround for ID753650' >> /config/startup # echo 'sysctl -w vm.min_free_kbytes=131072' >> /config/startup Note that the last two commands are not wrapped inside 'clsh' because the /config/startup file is already automatically synchronized across all blades. Once the issue is fixed in a future BIG-IP version, remove the workarounds: -- To remove the first workaround: 1) Edit the /etc/sysctl.conf file on all blades and remove the added lines at the bottom. 2) Reboot the system by running 'clsh reboot'. This will restore the min_free_kbytes kernel parameter to its default value for the BIG-IP version you are running. -- To remove the second workaround: 1) Edit the /config/startup file on the primary blade only, and remove the extra lines at the bottom. 2) Reboot the system by running 'clsh reboot'. This restores the min_free_kbytes kernel parameter to its default value for the BIG-IP version you are running. To verify the workaround is in place, run the following command (this should return the desired amount in KB): # clsh "cat /proc/sys/vm/min_free_kbytes"

Fix Information

None

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips