Bug ID 1623345: On rSeries system, power cycle reboot might result in QAT device failure

Last Modified: Jun 27, 2025

Affected Product(s):
F5OS F5OS-A(all modules)

Known Affected Versions:
F5OS-A 1.8.0

Opened: Aug 02, 2024

Severity: 3-Major

Symptoms

If you enable rate limiting and perform a power cycle reboot,it is possible that some or all of the QAT devices may not be operational. This is because of the sudden power shutdown. To check the rate-limiting status, use the following command. [root@appliance-1:Active] ~ # su admin <<< "show cluster nodes node node-1 state rate-limiting" state rate-limiting enabled [root@appliance-1:Active] ~ # For the problem occurred QAT devices the SLA allocation commands fail with below console messages when the hardware is coming up from power off/on reboot. You can also check these logs with dmesg command. [ 134.814182] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.820603] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.826998] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.833369] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.839754] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.846134] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.852479] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.858886] c6xx 0000:55:00.0: Service is not enabled 0 And console is flood with below du_mgr query logs. [ 5996.156402] c6xx 0000:54:00.0: Failed to query du VF: -22 [ 6001.748492] c6xx 0000:54:00.0: Service is not enabled 0 [ 6001.753717] c6xx 0000:54:00.0: Failed to query du VF: -22 [ 6007.351849] c6xx 0000:54:00.0: Service is not enabled 0 [ 6007.357079] c6xx 0000:54:00.0: Failed to query du VF: -22 [ 6012.965789] c6xx 0000:54:00.0: Service is not enabled 0 [ 6012.971001] c6xx 0000:54:00.0: Failed to query du VF: -22 [ 6018.853868] c6xx 0000:54:00.0: Service is not enabled 0 [ 6018.859096] c6xx 0000:54:00.0: Failed to query du VF: -22 [ 6024.585181] c6xx 0000:54:00.0: Service is not enabled 0 [ 6024.590401] c6xx 0000:54:00.0: Failed to query du VF: -22 [ 6030.197135] c6xx 0000:54:00.0: Service is not enabled 0 [ 6030.202348] c6xx 0000:54:00.0: Failed to query du VF: -22 And the confd table is not populated with rate limiting stats for the tenants deployed. [root@appliance-1:Active] ~ # su admin <<< " show cluster nodes node node-1 state cryptos " TENANT ASLA ASLA ASLA SLA SLA SLA NAME QAT DEVICE NAME BDF MIN USED UTIL MIN USED UTIL ----------------------------------------------------------------------- mytenant qat_dev_vf08pf04 c6:02.0 - - - - - - qat_dev_vf08pf05 c7:02.0 - - - - - - qat_dev_vf09pf00 53:02.1 - - - - - - qat_dev_vf09pf01 54:02.1 - - - - - - qat_dev_vf09pf02 55:02.1 - - - - - - qat_dev_vf09pf03 c5:02.1 - - - - - - qat_dev_vf09pf04 c6:02.1 - - - - - - qat_dev_vf09pf05 c7:02.1 - - - - - - qat_dev_vf10pf00 53:02.2 - - - - - - qat_dev_vf10pf01 54:02.2 - - - - - - [root@appliance-1:Active] ~ #

Impact

QAT devices may encounter an issue where Rate Limiting does not function properly for tenants deployed on the rSeries host. The Rate Limiting stats do not get updated in the ConfD and results in the console being flooded with error logs. The logs can also be viewed using the dmesg command. [ 5996.156402] c6xx 0000:54:00.0: Failed to query du VF: -22 [ 6001.748492] c6xx 0000:54:00.0: Service is not enabled 0 [ 6001.753717] c6xx 0000:54:00.0: Failed to query du VF: -22 [ 6007.351849] c6xx 0000:54:00.0: Service is not enabled 0

Conditions

The issue might occur in a power cycle reboot of the rSeries F5OS-A system with multiple tenants deployed. The below logs are observed on the console when this occurred. To check the logs, use the dmesg command. [ 134.814182] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.820603] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.826998] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.833369] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.839754] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.846134] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.852479] c6xx 0000:55:00.0: Service is not enabled 0 [ 134.858886] c6xx 0000:55:00.0: Service is not enabled 0

Workaround

Rebooting the appliance will solve the issue caused by the malfunctioning QAT devices.

Fix Information

None

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips