Bug ID 680850: Setting zxfrd log level to debug can cause AXFR and/or IXFR failures due to high CPU and disk usage.

Last Modified: Oct 01, 2018

Bug Tracker

Affected Product:  See more info
BIG-IP DNS(all modules)

Known Affected Versions:
12.1.0, 12.1.0 HF1, 12.1.0 HF2, 12.1.1, 12.1.1 HF1, 12.1.1 HF2, 12.1.2, 12.1.2 HF1, 12.1.2 HF2, 12.1.3, 12.1.3.1, 12.1.3.2, 12.1.3.3, 13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5

Fixed In:
14.0.0, 13.1.0.6, 12.1.3.4

Opened: Aug 30, 2017
Severity: 3-Major
Related AskF5 Article:
K48342409

Symptoms

Enabling debug logging on zxfrd (DNSX) can result in excessive CPU and disk usage, as well as errors during DNS AXFR or IXFR processing.

Impact

IXFRs or AXFRs may fail and be rescheduled due to high CPU usage by zxfrd, which causes it to fail to process data packets during a transfer.

Conditions

log.zxfrd.level is set to debug by running the following command: tmsh modify sys db log.zxfrd.level value debug

Workaround

To avoid this issue, do not set log.zxfrd.level to debug.

Fix Information

With this change, the dump to /var/tmp/zxfrd.out occurs only when a new db variable, dnsexpress.dumpastext, is set to true. This enables turning on logging for debug without consuming all the CPU and disk necessary to dump packets and zone contents. Note: Setting this value to true will cause the information to be dumped to stderr, which reveals the original issue, potentially causing transfer failures and high system resource usage. F5 recommends that you enable it only when directed to do so by F5 support staff.

Behavior Change

Previously, setting the zxfrd log level to debug caused all AXFR and IXFR requests and responses to be logged to /var/tmp/zxfrd.out. Doing so also caused the contents of all zones to be dumped, as text, to /var/tmp/zxfrd.out when the database was saved. This was extremely CPU-, memory-, and disk-intensive. The CPU load could cause zxfrd to fail to process transfer data packets in a timely fashion, which could cause the master DNS server to close the connection. With this fix, setting log.zxfrd.level debug no longer outputs this information. Although it is not generally useful to output the contents of the transfer packets or the contents of the database, if this information is required for troubleshooting or information-verification purposes, you can set the new db variable: dnsexpress.dumpastext. Note: Setting this value to true will cause the information to be dumped to stderr, which reveals the original issue, potentially causing transfer failures and high system resource usage. F5 recommends that you enable it only when directed to do so by F5 support staff.