Last Modified: Feb 11, 2025
Affected Product(s):
BIG_IP_NEXT(CM) TMOS
Known Affected Versions:
20.0.1, 20.0.2, 20.1.0, 20.1.1, 20.2.0
Fixed In:
20.2.1
Opened: Apr 03, 2024 Severity: 2-Critical
In some cases, you might be unable to complete the BIG-IP Next Central Manager initial setup (script) due to fluentd in CrashLoopBackOff and script timing out. var/log/central-manager/central-manager-cli.log Error: INSTALLATION FAILED: client rate limiter Wait returned an error: context deadline exceeded var/log/syslog Mar 7 14:08:11 central-manager k3s[1940]: E0307 14:08:11.499777 1940 pod_workers.go:1294] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"fluentd\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=fluentd pod=mbiq-fluentd-0_default(fa83f8e7-9ffd-451e-912e-f6fb037a083d)\"" pod="default/mbiq-fluentd-0" podUID=fa83f8e7-9ffd-451e-912e-f6fb037a083
You may be unable to complete the initial setup of the BiG-Ip Next Central Manager.
The BIG-IP Next Central Manager Virtual Machine is deployed on a hypervisor with high resource utilization (eg long/slow storage response).
- Make sure you start with a configuration with default numbers of vCPUs. - Run uninstall: /opt/cm-bundle/cm uninstall - Start a new setup, fill in all details: setup - When the installer asks the following, stop for a moment. Would you like to start the BIG-IP Next Central Manager application installation (Y/n) [Y]: - Open a new ssh session and navigate to: cd /var/opt/cm-bundle/artifacts/ - Create a new directory and unpack umbrella spec to it: mkdir custom tar -C custom -zxvf umbrella-20.1.1-1.1.tgz - Save the old umbrella spec: mv umbrella-20.1.1-1.1.tgz umbrella-20.1.1-1.1.tgz.old - Open fluentd chart: sudo chmod 644 custom/umbrella/charts/fluentd/values.yaml vi custom/umbrella/charts/fluentd/values.yaml - Within the chart navigate to startup probe section. NOTE! There are two sections like this you need to modify: startupProbe: enabled: false httpGet: path: /fluentd.healthcheck?json=%7B%22ping%22%3A+%22pong%22%7D port: http initialDelaySeconds: 60 periodSeconds: 10 timeoutSeconds: 5 failureThreshold: 6 successThreshold: 1 - Give it some more time, edit initialDelaySeconds to 180 (or periodSeconds). NOTE! There are two sections like this you need to modify. Save changes. initialDelaySeconds: 180 - Tar the new file from /custom dir (make sure file structure remains the same as in the original file): root@lab:/var/opt/cm-bundle/artifacts/custom# tar -cvzf ../umbrella-20.1.1-1.1.tgz umbrella/ - From the previous ssh window, proceed with installation: Would you like to start the BIG-IP Next Central Manager application installation (Y/n) [Y]: Y - When fluentd install completes, from that separate SSH session, see if changes were applied successfully, you should see a similar section to the one you edited before: k edit pod mbiq-fluentd-0
The Fluentd configuration has been updated to provide additional time for the initial startup.