Bug ID 766577: APMD fails to send response to client and it already closed connection.

Last Modified: Nov 18, 2019

Bug Tracker

Affected Product:  See more info
BIG-IP APM(all modules)

Known Affected Versions:
12.1.0, 12.1.0 HF1, 12.1.0 HF2, 12.1.1, 12.1.1 HF1, 12.1.1 HF2, 12.1.2, 12.1.2 HF1, 12.1.2 HF2, 12.1.3, 12.1.3.1, 12.1.3.2, 12.1.3.3, 12.1.3.4, 12.1.3.5, 12.1.3.6, 12.1.3.7, 12.1.4, 12.1.4.1, 13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 13.1.1.5, 13.1.3, 13.1.3.1, 14.0.0, 14.0.0.1, 14.0.0.2, 14.0.0.3, 14.0.0.4, 14.0.0.5, 14.0.1, 14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.4, 14.1.0.5, 14.1.0.6, 14.1.2, 14.1.2.1, 14.1.2.2, 15.0.0, 15.0.1

Fixed In:
14.0.1.1, 13.1.3.2, 12.1.5

Opened: Mar 28, 2019
Severity: 3-Major

Symptoms

APMD fails to send response to client and produces error message: err apmd[8353]: 01490085:3: /pt-qp-campus/apm-cdp-qp-qa:pt-qp-campus:bb651ae6: Response could not be sent to remote client. Socket error: Connection reset by peer APMD does most of its action with backend authentication servers (e.g., AD, LDAP, RADIUS). If the backend server response is very slow (because of various reasons such as network issues), it might cause slow apmd client response. Sometimes, the client has already closed the connection to the virtual server, so the client connection is no longer valid.

Impact

This causes the client to close the connection. APMD fails to respond to the client. The cumulative slowness of the backend server causes delay in response. Most of the time, the client connection is already closed. As a result, the request queue gets full. When apmd starts processing the request from the queue, the client connection is already closed for some of them, and processing those requests still continues, which is unnecessary and causes more delay.

Conditions

Backend server is slow, causing longer-than-usual response times.

Workaround

None

Fix Information

The system now tests the client connection after picking up the request from the request queue and before processing. -- If the connection is already closed, the system drops the request. -- If the request is already in progress, the system checks the client connection before saving the session variables and sending the response to client.

Behavior Change