Last Modified: Apr 28, 2025
Affected Product(s):
BIG-IP LTM
Known Affected Versions:
13.0.1, 13.0.0, 12.1.3, 12.1.2, 12.1.1, 12.1.0, 12.0.0, 11.6.5, 11.6.3, 11.6.2, 11.6.1, 11.6.0, 11.5.9, 11.5.8, 11.5.7, 11.5.6, 11.5.5, 11.5.4, 11.5.3, 11.5.2, 11.5.10, 11.5.1, 11.5.0, 11.4.1, 11.4.0, 11.3.0, 11.2.1, 11.2.0
Fixed In:
13.1.0, 12.1.3.4
Opened: May 23, 2013 Severity: 3-Major
For several seconds after a failover of redundant BIG-IP units, some application traffic can still be seen egressing the newly standby unit.
The offloaded flows are eventually evicted from the ePVA after the failover switch period (16 seconds by default). However, traffic in some deployments may be impacted during this time. What determines whether any impact occurs or not is a combination of: -- Whether MAC masquerading is used or not. -- How many IP addresses need to be advertised after a failover. -- The speed and efficiency with which surrounding network equipment learns the IP addresses have moved. -- Whether hardware offloaded flows and regular software flows share a common virtual address. This matters because even though a hardware offloaded flow can continue to work despite being processed by the standby unit, egressing such packets may cause a MAC address change and/or port movement for a specific IP address on the surrounding network equipment. In turn, this may cause regular software flows for the virtual address to be directed to the standby unit, and these flows fail as TMM does not process them in standby mode.
This issue occurs when all of the following conditions are met: -- BIG-IP units are deployed in a redundant configuration. -- BIG-IP units are hardware models with the ePVA chip. -- One or more virtual servers employ hardware acceleration. -- A failover of the units occurs. -- For some specific reason, the newly standby unit still receives some application traffic (matching hardware offloaded flows) from the network. It is this traffic which continues to be processed (and egressed back to the network) by the ePVA chip.
There are three possible mitigation scenarios: -- Reduce the likelihood of this issue by configuring MAC masquerading for your Traffic-Groups. This can improve the situation, as the surrounding network equipment no longer has to learn a MAC address change for a given IP address during a failover of the BIG-IP devices (the equipment only has to learn the port move for the MAC masquerade address). -- Ensure the surrounding network equipment is not limiting GARPs (or broadcast ARP requests/responses in general). -- Employ the 'link down time on failover' feature to keep links down after a failover for a few seconds (thus preventing the possibility of any traffic egressing the standby unit). This is not possible for vCMP guests, however.
There is now a database variable 'pva.standby.flush' which can be enabled to flush accelerated flows after a failover. Setting 'pva.standby.flush' to 1 instructs the BIG-IP system to evict all accelerated flows from the ePVA hardware after it registers a failover event. Note: If multiple Traffic-Groups are configured, a failover from any of them causes this flush, regardless of the state of any other Traffic-Groups. -- To enable this functionality: tmsh modify sys db pva.standby.flush value 1 -- To disable this functionality (the default setting): tmsh modify sys db pva.standby.flush value 0 Note: The term 'flush' or 'eviction' does not mean flow 'deletion'; flow eviction occurs normally in other situations. The impact from the flush/eviction of flows on other, still active Traffic-Groups results in brief, increased resource consumption until the flows are returned to ePVA acceleration.