Bug ID 1694481: K3s token expiry causing tenant unresponsiveness

Last Modified: May 01, 2025

Affected Product(s):
F5OS F5OS-A(all modules)

Known Affected Versions:
F5OS-A 1.5.1, F5OS-A 1.5.2, F5OS-A 1.7.0, F5OS-A 1.8.0

Fixed In:
F5OS-A 1.5.3

Opened: Oct 10, 2024

Severity: 2-Critical

Symptoms

Expiry of service account token inside multus causes the tenant to become unresponsive. Note: It will only be impacted if/after it is changed to configured or provisioned and then it is deployed again. The tenant fails to come up and the tenant status reads: #show tenants; tenant STATUS Not ready: containers with unready status: [compute]

Impact

After one year, token in the multus.kubeconfig becomes stale (expired). As a result, when Multus tries to access the Kubernetes API server using the stale token in the multus.kubeconfig, it may fail with authentication errors because the token is no longer valid.

Conditions

-- Multus.kubeconfig is not recreated or updated when the service account token in /var/run/secrets/kubernetes.io/serviceaccount/token is renewed. -- Even though the token is renewed, the token is still valid for a year in multus.kubeconfig

Workaround

Workaround(1): Impact of procedure: Performing the following procedure should not have a negative impact on your system. Delete the multus pod by logging into the system as root and running the following command: kubectl -n kube-system delete pod -l app=multus The system will delete the running pod and create a new one. This will refresh the token for the next one year. Workaround(2): Impact of procedure: Tenants will be temporarily unavailable during this process. Rebooting the device will refresh the token.

Fix Information

None

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips