Bug ID 725791: Potential HW/HSB issue detected

Last Modified: Sep 19, 2019

Bug Tracker

Affected Product:  See more info
BIG-IP LTM(all modules)

Known Affected Versions:
11.5.1, 11.5.1 HF1, 11.5.1 HF10, 11.5.1 HF11, 11.5.1 HF2, 11.5.1 HF3, 11.5.1 HF4, 11.5.1 HF5, 11.5.1 HF6, 11.5.1 HF7, 11.5.1 HF8, 11.5.1 HF9, 11.5.2, 11.5.2 HF1, 11.5.3, 11.5.3 HF1, 11.5.3 HF2, 11.5.4, 11.5.4 HF1, 11.5.4 HF2, 11.5.4 HF3, 11.5.4 HF4, 11.5.5, 11.5.6, 11.5.7, 11.5.8, 11.5.9, 11.6.0, 11.6.0 HF1, 11.6.0 HF2, 11.6.0 HF3, 11.6.0 HF4, 11.6.0 HF5, 11.6.0 HF6, 11.6.0 HF7, 11.6.0 HF8, 11.6.1, 11.6.1 HF1, 11.6.1 HF2, 11.6.2, 11.6.2 HF1, 11.6.3, 11.6.3.1, 11.6.3.2, 11.6.3.3, 11.6.3.4, 11.6.4, 11.6.5, 12.0.0, 12.0.0 HF1, 12.0.0 HF2, 12.0.0 HF3, 12.0.0 HF4, 12.1.0, 12.1.0 HF1, 12.1.0 HF2, 12.1.1, 12.1.1 HF1, 12.1.1 HF2, 12.1.2, 12.1.2 HF1, 12.1.2 HF2, 12.1.3, 12.1.3.1, 12.1.3.2, 12.1.3.3, 12.1.3.4, 12.1.3.5, 12.1.3.6, 12.1.3.7, 12.1.4, 12.1.4.1, 12.1.5, 13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 14.0.0, 14.0.0.1, 14.0.0.2, 14.0.0.3, 14.0.0.4, 14.0.0.5, 14.0.1, 14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.4, 14.1.0.5

Fixed In:
15.0.0, 14.1.0.6, 13.1.1.5

Opened: Jun 26, 2018
Severity: 3-Major

Symptoms

There are a number of High-Speed Bridge (HSB) stats registers that monitor the errors in HSB SRAM that are critical for passing traffic, for example, RQM_CRC_ERROR Count 0, RQM_CRC_ERROR count 1, RQM_CRC_ERROR Count 2, etc. Any errors in any of these registers may indicate a hardware error in the HSB SRAM that impedes traffic through embedded Packet Velocity Acceleration (ePVA). In that case, ePVA-accelerated flow might fail. With a burst of CRC errors in the SRAM for ePVA transformation cache, it does not trigger a failover and causes a silent traffic outage on the FastL4 VIP with hardware traffic acceleration. This occurs because the health check watchdog packets are still functioning correctly, and the current TMOS software primarily monitors watchdog packets tx/rx failures to trigger failover. In these cases, there might be the following messages in /var/log/tmm*: Device error: hsb_lbb* tre2_crc_errs count *

Impact

Hardware accelerated traffic drop.

Conditions

Traffic is offloaded to HSB hardware for acceleration.

Workaround

Switch traffic to software acceleration.

Fix Information

Including traffic-critical registers in failover triggers, helps failover happen quickly with minimum disruption to traffic in the case of SRAM hardware failures.

Behavior Change