Bug ID 910213: LB::down iRule command is ineffective, and can lead to inconsistent pool member status

Last Modified: Jul 21, 2021

Bug Tracker

Affected Product:  See more info
BIG-IP LTM(all modules)

Known Affected Versions:
16.1.0, 16.0.1, 16.0.0, 15.1.3, 15.1.2, 15.1.1, 15.1.0, 15.0.1, 15.0.0, 14.1.4, 14.1.3, 14.1.2.5, 14.1.2.4, 14.1.2.3, 14.1.2.2, 14.1.2.1, 14.1.0, 14.0.1, 14.0.0, 13.1.4, 13.1.3.3, 13.1.3.2

Opened: May 19, 2020
Severity: 2-Critical

Symptoms

Use of the LB::down command in an iRule may not have the desired effect, or may result in pool members that are down for load balancing, but indicate up/available in the GUI and CLI. Specifically, the pool member is marked down within the tmm instance executing the iRule, but the status change is not updated to mcpd, or to other tmm instances. As a result, the message 'Pool /Common/mypool member /Common/1.1.1.1:80 monitor status iRule down' does not appear in the log, and the status of the pool member is not updated when viewed in the GUI or via 'tmsh show ltm pool xxxx members'. Note: If [event info] is logged in the LB_FAILED event, it will indicate that the load balancing decision failed due to "connection limit"

Impact

Because mcpd believes the pool member to be up, it does not update tmm's status, so tmm continues to regard it as down indefinitely, or until a monitor state change occurs. If the LB::down command is used on all members of a pool, the affected tmms cannot load balance to that pool, even though the GUI/tmsh indicate that the pool has available members. Because pool member status is stored on per-tmm basis and incoming connections are distributed across tmms using a hash, this can lead to apparently inconsistent results, where some traffic (traffic hitting a particular tmm) is rejected with an RST cause of 'No pool member available'.

Conditions

Using the LB::down command in an iRule.

Workaround

- Delete and recreate affected pool members (or) Restart tmm (or) Restart the BIG-IP. There is no direct workaround, but the use of an inband monitor instead of the LB::down command may be effective. You must tune the inband monitor's settings to values consistent with the desired behavior.

Fix Information

None

Behavior Change