Bug ID 769581: Timeout when sending many large iControl REST requests

Last Modified: Jul 12, 2023

Affected Product(s):
BIG-IP TMOS(all modules)

Known Affected Versions:
13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.2, 13.1.1.3, 13.1.1.4, 13.1.1.5, 13.1.3, 13.1.3.1, 13.1.3.2, 13.1.3.3, 13.1.3.4, 14.0.0, 14.0.0.1, 14.0.0.2, 14.0.0.3, 14.0.0.4, 14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.5, 14.1.0.6, 14.1.2, 14.1.2.1, 14.1.2.2, 14.1.2.3, 14.1.2.4, 14.1.2.5, 14.1.2.6, 15.0.0, 15.0.1, 15.0.1.1, 15.0.1.2, 15.0.1.3, 15.0.1.4

Fixed In:
15.1.0, 14.1.2.7, 14.0.0.5, 13.1.3.5

Opened: Apr 05, 2019

Severity: 2-Critical

Symptoms

After sending hundreds of REST requests, REST requests eventually begin to time out. This is the case for applications such as an AS3, with requests with 700 services.

Impact

Saving new configuration data does not work. Any new transaction tasks fail.

Conditions

1. Download and install the AS3 iApp. This adds the /mgmt/shared/appsvcs/ endpoint to the the BIG-IP system. 2. Deploy config with AS3: curl -X POST \ https://<$IP_address>/mgmt/shared/appsvcs/declare \ -H 'Content-Type: application/json' \ -d //This should be the data from an AS3 body 3. While deployment in step 2 is happening, make a GET to the tasks: curl -X GET \ https://<$IP_address>/mgmt/shared/appsvcs/task \ -H 'Content-Type: application/json' 4. Delete configuration: curl -X DELETE \ https://<$IP_address>/mgmt/shared/appsvcs/declare It may take 3 or 4 times repeating steps 2 through 4 for the issue to show up. When it appears, you will start seeing messages in the AS3 task response like the following: -- 'message': 'failed to save BIG-IP config (POST http://<$USERNAME>:<$PASSWORD>@<$IP_address>:8100/mgmt/tm/task/sys/config create task save sys config response=400 body={\"code\":400,\"message\":\"remoteSender:Unknown, method:POST \",\"originalRequestBody\":\"{\\\"command\\\":\\\"save\\\"}\",\"referer\":\"Unknown\",\"restOperationId\":6924816,\"kind\":\":resterrorresponse\"})'

Workaround

1. Restart restjavad and all iControl REST (icrd_child) instances: #tmsh restart sys service restjavad 2. Wait longer for large requests to finish before performing additional requests.

Fix Information

The system now handles the new transaction iControl REST creation process properly when the existing process was killed with a timeout operation.

Behavior Change

Guides & references

K10134038: F5 Bug Tracker Filter Names and Tips