Bug ID 2216921: Controller can incorrectly be removed from the OpenShift cluster during rolling upgrade

Last Modified: Oct 06, 2026

Affected Product(s):
F5OS F5OS-C, Install/Upgrade(all modules)

Known Affected Versions:
F5OS-C 1.8.2

Fixed In:
F5OS-C 1.8.2-HR1, F5OS v2.0.0

Opened: Jan 25, 2026

Severity: 2-Critical

Symptoms

After a rolling upgrade of the controller software, one of the controllers may incorrectly be removed from the OpenShift cluster. This leaves the OpenShift cluster in a state where there is only one leader node in the cluster.

Impact

If this occurs the affected controller will no longer be in the OpenShift cluster, so HA availability of the OpenShift cluster between the controllers will be compromised.

Conditions

This can happen during a rolling upgrade due to a race condition on checking the status of the standby controller.

Workaround

To protect against a controller being removed from the cluster during the rolling upgrade while the active controller is still running the older software, it is recommended to do a "touch /tmp/omd/pause" on the active controller before starting the rolling upgrade. This will keep the down rev controller from removing the newly upgraded controller if the bug is hit. The pause file be automatically removed when the controller is upgraded after the newly upgraded controller becomes active. The workaround to recover once the controller has been removed from the cluster is to manually retrigger the addition of the removed controller to the Openshift cluster. If controller-1 was removed, on controller 2 the user would do a touch /var/omd/CONTROLLER1_REINSTALL If controller-2 was removed, on controller 1 the user would do a touch /var/omd/CONTROLLER2_REINSTALL

Fix Information

The race condition that led to the controller being incorrectly removed from the cluster has been fixed, so that a controller will no longer be removed from the cluster during rolling upgrade once the controllers have been upgraded.

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips