Last Modified: Apr 19, 2025
Affected Product(s):
F5OS F5OS-A
Known Affected Versions:
F5OS-A 1.5.3
Opened: Nov 05, 2024 Severity: 3-Major
After a power cycle restart of the F5OS-A rSeries system, existing tenant pods could go into a restart loop and never become active. Newly created tenants may also fail to start. vfio reports Device or resource busy. The tenant pod state can be checked with the below command on the host system. [root@appliance-1:Active] vfio # kubectl get pods NAME READY STATUS RESTARTS AGE f5-resource-manager-bpnrr 1/1 Running 0 3h virt-launcher-bigip-14-1-kz56l 1/1 Running 0 3h4m virt-launcher-bigip-19-1-5m72j 1/1 Running 0 3h4m virt-launcher-bigip-3-1-pn6c2 1/1 Running 0 3h4m virt-launcher-bigip-4-1-8x4cc 1/1 Running 0 3h4m virt-launcher-bigip-20-1-q99b7 1/1 Running 0 3h4m virt-launcher-bigip-5-1-vr4cf 1/1 Running 0 3h4m virt-launcher-bigip-18-1-zfrns 1/1 Running 0 162m virt-launcher-bigip-1-1-qhjd5 1/1 Terminating 0 4m8s virt-launcher-bigip-13-1-vjwwd 1/1 Terminating 0 3m19s virt-launcher-bigip-12-1-7swfq 0/1 Completed 0 87s virt-launcher-bigip-16-1-pqjx6 1/1 Running 0 43s virt-launcher-bigip-15-1-56x2g 0/1 PodInitializing 0 5s [root@appliance-1:Active] vfio #
Some or all of the vfio devices could run into the problem, which results in some or all tenants deployed on the rSeries host do not work as expected. They do not change to a RUNNING state.
The issue might occur in a power cycle restart, live software upgrade, or any operation that involves a reboot of the rSeries F5OS-A system. The below logs will be observed. [root@appliance-1:Active] # kubectl get pods This command shows the pod name. You can use the following command to see the log in the problem pod. Hash in the pod name changes for every restart of the pod. [root@appliance-1:Active] # kubectl logs <Problem Pod name displayed> | grep busy qemu-kvm: -device vfio-pci,host=0000:53:01.6,id=hostdev0,bus=pci.10,addr=0x0: vfio 0000:53:01.6: failed to open /dev/vfio/95: Device or resource busy
As the vfio devices are in a problem state, a graceful reboot of the appliance will resolve the issue.
None