Bug ID 1035121: Configsync syncs the node's monitor status

Last Modified: Jan 20, 2023

Bug Tracker

Affected Product:  See more info
BIG-IP LTM(all modules)

Known Affected Versions:
12.1.0, 12.1.0 HF1, 12.1.0 HF2, 12.1.1, 12.1.1 HF1, 12.1.1 HF2, 12.1.2, 12.1.2 HF1, 12.1.2 HF2, 12.1.3, 12.1.3.1, 12.1.3.2, 12.1.3.3, 12.1.3.4, 12.1.3.5, 12.1.3.6, 12.1.3.7, 12.1.4, 12.1.4.1, 12.1.5, 12.1.5.1, 12.1.5.2, 12.1.5.3, 12.1.6, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 13.1.1.5, 13.1.3, 13.1.3.1, 13.1.3.2, 13.1.3.3, 13.1.3.4, 13.1.3.5, 13.1.3.6, 13.1.4, 13.1.4.1, 13.1.5, 13.1.5.1, 14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.5, 14.1.0.6, 14.1.2, 14.1.2.1, 14.1.2.2, 14.1.2.3, 14.1.2.4, 14.1.2.5, 14.1.2.6, 14.1.2.7, 14.1.2.8, 14.1.3, 14.1.3.1, 14.1.4, 14.1.4.1, 14.1.4.2, 14.1.4.3, 14.1.4.4, 14.1.4.5, 14.1.4.6, 14.1.5, 14.1.5.1, 14.1.5.2, 14.1.5.3, 15.1.0, 15.1.0.1, 15.1.0.2, 15.1.0.3, 15.1.0.4, 15.1.0.5, 15.1.1, 15.1.2, 15.1.2.1, 15.1.3, 15.1.3.1, 15.1.4, 15.1.4.1, 15.1.5, 15.1.5.1, 15.1.6, 15.1.6.1, 15.1.7, 15.1.8, 15.1.8.1, 16.1.0, 16.1.1, 16.1.2, 16.1.2.1, 16.1.2.2, 16.1.3, 16.1.3.1, 16.1.3.2, 16.1.3.3

Fixed In:
17.0.0

Opened: Jul 21, 2021
Severity: 2-Critical

Symptoms

After config sync, nodes may be marked marked down when they are up, even if the monitor determines that the node is up. The logs will show something similar to : notice mcpd[8091]: 010714a0:5: Sync of device group /Common/device_trust_group to commit id 1 6986973310536375596 /Common/xxxxxxxx 1 from device /Common/yyyyyyyy notice mcpd[8091]: 01070640:5: Node /Common/node1 address 10.10.100.1 monitor status down. [ /Common/icmp: up ] [ was up for 0hr:3mins:15sec ] notice mcpd[8091]: 01070640:5: Node /Common/node2 address 10.10.100.2 monitor status down. [ /Common/icmp: up ] [ was up for 0hr:3mins:15sec ] The node/pool member/pool/virtual server will be marked down. Checking the actual monitor it will be up, tcpdump will show successful monitor transactions.

Impact

The node's monitor status is synced to the peer device. If the from-device's monitor was unable to reach the nodes and was marking the nodes as DOWN, then the node status will be set to DOWN on the other device, even if the monitor is successfully connecting to the node. This can cause a traffic disruption. Note: the opposite can occur, where a "node up" status is sent to a device whose monitor is failing to connect to the nodes due to a network issue.

Conditions

1. Two or more devices in a sync/failover device group 2. The config sync from-device has marked nodes down 3. A config sync occurs This can occur on both incremental and full config sync.

Workaround

If a device is in this state, you can work around this issue by doing one of the following: -- Save and reload the configuration on the device with the bad state tmsh save sys config && tmsh load sys config -- Perform a full-load sync from the peer device to the affected device: (On the peer) tmsh run cm config-sync force-full-load-push to-group group-name

Fix Information

Node monitor statuses are not synced between devices.

Behavior Change