Bug ID 1410609: Watchdog resets during PSU management may cause AOM/LOP to remain in bootloader mode

Last Modified: Jan 30, 2025

Affected Product(s):
F5OS AOM, F5OS-C, Velos(all modules)

Fixed In:
F5OS-C 1.8.0

Opened: Dec 05, 2023

Severity: 3-Major

Symptoms

The system controller AOM/LOP may encounter a watchdog reset while doing PSU management. If multiple watchdog resets occur in succession, then the AOM/LOP may remain in bootloader mode and be unavailable. When this occurs, the PEL log will indicate a LOP watchdog reset in the LopPsuManagement task, for example: 07/17/2024 06:27:14 | 36644 | AOM | 128 | Network Access | 5 | LopPsuManagement task 100% of watchdog period, resettin 07/17/2024 06:27:20 | 36645 | AOM | 190 | Network Access | 5 | watchdog reset, successive watchdog resets: 9 When there have been 10 successive watchdog resets then AOM/LOP remains in bootloader mode and needs to be reprogrammed.

Impact

If multiple watchdog resets occur in succession, then the system controller's AOM/LOP may remain in bootloader mode and be unavailable.

Conditions

- A VELOS system controller, in either the CX-410 or CX-1610 chassis.

Workaround

At the system controller host prompt, verify that the AOM/LOP is in bootloader mode. [root@controller-1 ~]# lsusb Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 002: ID f5f5:df11 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub An AOM/LOP in bootloader mode will enumerate as USB device f5f5:df11 as shown above. To reprogram the AOM/LOP firmware, first locate the latest firmware version provided by F5OS-C. For example: [root@controller-1 ~]# ls $(docker container inspect platform-fwu -f '{{ range.Mounts}}{{.Source}}{{printf "\n"}} {{end}}' | grep config_fw-volume) | grep ^lop-chassis-controller lop-chassis-controller-v2.01.1238.0.1.dfu Then reprogram the AOM/LOP using the firmware version located above, for example: [root@controller-1 ~]# docker exec -it platform-fwu dfu-util -D /usr/lib/firmware/lop-chassis-controller-v2.01.1238.0.1.dfu It will take approximately 2 minutes to reprogram the AOM/LOP firmware image. The AOM/LOP will enumerate as USB device f5f5:3000 after reprogramming, as shown: [root@controller-1 ~]# lsusb Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 002: ID f5f5:3000 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Fix Information

Fixed in lop-chassis-controller-application-2.01.1276.0.1 and later.

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips