Bug ID 1576545: After upgrade, BIG-IP Next tenant os unable to export toda-otel (event logs) data to Central Manager

Last Modified: Apr 28, 2025

Affected Product(s):
F5OS Install/Upgrade(all modules)

Fixed In:
F5OS-C 1.8.0

Opened: Apr 08, 2024

Severity: 1-Blocking

Symptoms

After upgrade, the BIG-IP Next tenant is unable to export toda-otel (event logs) data to CM in VELOS

Impact

After upgrade, the BIG-IP Next tenant is unable to export toda-otel (event logs) data to CM

Conditions

Upgrading BIG-IP Next tenant from 20.1 to 20.2 on a VELOS system.

Workaround

For VELOS Standalone ==================== After upgrade, if the f5-toda-otel-collector cannot connect to host change the tenant status from "DEPLOYED" TO "CONFIGURED" TO "DEPLOYED" to fix the issue. Please note that it will take 5 to 10 min for tenant status to change and it might impact the traffic. For VELOS HA follow the following steps ======================================= 1. Setup CM on Mango build 2. Add 2 BIG-IP Next instances(Mango build) on the CM 3. Bring up HA on CM with the Enable Auto Failover option unchecked 4. Add a license to the HA instance. 5. Deploy a basic HTTP app in FAST mode with WAF policy attached (Enforcement mode - Blocking, Log Events - all) 6. Send the traffic and verify the WAF Dashboard under the Security section, should be able to see the Total Requests and Blocked response fields with non-zero values 7. Upgrade standby instance to latest nectarine build with the "auto-failover" button switched off. 8. We will observe the instances goes into an unhealthy state on CM. 9. Change the status of the standby instance from Deployed to Configure Mode and save it through partition GUI/CLI. 10. After confirming the status of the pods, change the state of the standby instance back to the Deployed state from the configured state. There should be no impact on the traffic flow during this step. 11. Now do the force failover and check the health status of instances, it will still show unhealthy as instances are in between upgrades.(one instance with Mango build (standby node) and other with Nectarine build(Active node)) 12. Now Upgrade the standby instance to the latest nectarine build with the "auto-failover" button switched off. 13. HA should look healthy in this state and traffic should continue to flow. 14. Change the state of the standby instance from Deployed to Configure Mode and save it using partition GUI/CLI 15. After confirming the status of the pods for the instance on partition CLI, change the state of the standby instance back to the Deployed state from the configured state. 16. We will observe the Event logs on the WAF Dashboard under the security section on CM. 17. We can also observe the logs on the "f5-toda-otel-collector" pod showing no Export failures. 18. Upgrade the CM. Systems should be Healthy.

Fix Information

None

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips