Bug ID 489562: HTTP with NTLMSSP_NEGOTIATE message and with payload more than 4KB cause the NTLM front end authentication to stall

Last Modified: Oct 17, 2023

Affected Product(s):
BIG-IP APM(all modules)

Known Affected Versions:
11.3.0, 11.6.0, 11.6.1, 11.6.2, 11.6.3, 11.6.3.1, 11.6.3.2, 11.6.3.3, 11.6.3.4, 11.6.4, 11.6.5, 11.6.5.1, 11.6.5.2, 11.6.5.3, 12.0.0, 12.0.0 HF1, 12.1.0 HF1, 12.0.0 HF2, 12.1.0 HF2, 12.0.0 HF3, 12.0.0 HF4, 12.1.1 HF1, 12.1.1 HF2, 12.1.2 HF1, 12.1.2 HF2, 12.1.0, 12.1.1, 12.1.2, 12.1.3, 12.1.3.1, 12.1.3.2, 12.1.3.3, 12.1.3.4, 12.1.3.5, 12.1.3.6, 12.1.3.7, 12.1.4, 12.1.4.1, 12.1.5, 12.1.5.1, 12.1.5.2, 12.1.5.3, 12.1.6, 13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 13.1.1.5, 13.1.3, 13.1.3.1, 13.1.3.2, 13.1.3.3, 13.1.3.4, 13.1.3.5, 13.1.3.6, 13.1.4, 13.1.4.1, 13.1.5, 13.1.5.1, 14.0.0, 14.0.0.1, 14.0.0.2, 14.0.0.3, 14.0.0.4, 14.0.0.5, 14.0.1, 14.0.1.1

Opened: Nov 08, 2014

Severity: 3-Major

Symptoms

NTLM authentication cannot be completed in the following circumstances. It is observed that some non-Microsoft HTTP clients might start NTLM authentication by sending a NTLMSSP_NEGOTIATE message together with a payload. As part of NTLM protocol, the response to this request should be a 401 status with an NTLMSSP_CHALLENGE message which renders the payload from the initial request unnecessary. However, the issue is that currently the BIG-IP system has a limit of 4KB for initial buffer, and does not drop it. This causes a deadlock between the BIG-IP server and HTTP client, as the BIG-IP notifies the client that it cannot receive the payload any more by closing the TCP receive window, and the client tries to complete sending all of the requests to be able to send the final NTLMSSP_AUTHENTICATE message.

Impact

NTLM authentication cannot be completed.

Conditions

The client sends NTLMSSP_NEGOTIATE message with payload of more than 4KB and the BIG-IP system performs NTLM authentication for this request.

Workaround

None

Fix Information

None

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips