Bug ID 760680: TMSH may utilize 100% CPU (single core worth) when set to be a process group leader and SSH session is closed.

Last Modified: Oct 28, 2019

Bug Tracker

Affected Product:  See more info
BIG-IP TMOS(all modules)

Known Affected Versions:
14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.4, 14.1.0.5, 14.1.0.6, 14.1.2, 15.0.0, 15.0.1

Fixed In:
14.1.2.1

Opened: Mar 07, 2019
Severity: 3-Major

Symptoms

TMSH does not correctly handle absence of input stream after closing interactive SSH session and remains active in an infinite loop using 100% CPU.

Impact

The equivalent of one CPU core is utilized to 100% by the TMSH process. It may be mostly scheduled on one core or spread over multiple control plane cores.

Conditions

If TMSH is a process group leader, it will not be killed when the parent shell is terminated upon SSH session close. This is a rare case, as TMSH must be deliberately promoted to a process group leader, e.g., with the 'setsid' command. Usually the shell process is a group leader and, when it is terminated upon SSH session close, it kills its child processes, including TMSH.

Workaround

TMSH should not be intentionally promoted to a process group leader. You can kill all TMSH processes using the command: killall -9 tmsh Warning: This command kills both abandoned and in-use TMSH processes. The latter can include other users' TMSH shells, and even system-level processes invoking the TMSH utility internally. Killing all TMSH processes can lead to various unexpected failures. You can use the 'top' command to see which TMSH process is using high CPU (e.g., 90% or more), and kill just those, as those are the likely zombie processes. You can kill specific TMSH processes using the command: kill -9 <pid> Where <pid> is the process ID of the TMSH instance to kill.

Fix Information

I/O error handling in TMSH has been corrected, so it no longer ignores absence of input stream which led to infinite loop.

Behavior Change