Bug ID 904657: HTTP monitors containing line breaks

Last Modified: Nov 23, 2020

Bug Tracker

Affected Product:  See more info
BIG-IP All, Install/Upgrade(all modules)

Known Affected Versions:
14.1.0, 14.1.0.1, 14.1.0.2, 14.1.0.3, 14.1.0.5, 14.1.0.6, 14.1.2, 14.1.2.1, 14.1.2.2, 14.1.2.3, 14.1.2.4, 14.1.2.5, 14.1.2.6, 14.1.2.7, 14.1.2.8, 15.0.0, 15.0.1, 15.0.1.1, 15.0.1.2, 15.0.1.3, 15.0.1.4, 15.1.0, 15.1.0.1, 15.1.0.2, 15.1.0.3, 15.1.0.4, 15.1.0.5, 15.1.1, 15.1.2

Opened: Apr 28, 2020
Severity: 3-Major

Symptoms

BIG-IP HTTP monitors containing line breaks can cause some of the following symptoms: a. Import errors: When importing BIG-IP devices with such these monitors, BIG-IQ returns a message similar to the following: "HTTP monitor <monitor_name> send contains new line please represent new line as '\n' instead." The message also contains the property at fault (send, receive, etc.). b. Limited support available in BIG-IP user interface: You can edit these monitors from the BIG-IP user interface but when you save the changes the line breaks are removed. You edit these line breaks from the command line. c. Upgrade errors: BIG-IP containing monitors with line breaks can occasionally cause issues during BIG-IP upgrades.

Impact

You might not be able to import the BIG-IP device to BIG-IQ. You might not be able to modify the monitor through the BIG-IP user interface. You might run into errors while upgrading the BIG-IP device.

Conditions

This happens if a BIG-IP configuration includes a HTTP monitor with strings containing unsupported characters within the string values for 'send' or 'receive' properties. The unsupported character is termed a 'physical line break': a string split across multiple lines, possibly created by using the 'enter key'. For example, "line1 line2" A newline character '\n' is a supported character and is treated differently from the 'physical line break'. This happens when you have monitors with line breaks.

Workaround

To work around this, update the monitor to remove the line breaks as follows. 1. Log on to the BIG-IP system's user interface. 2. Open the monitor. 3. Add '\n' characters to replace the line breaks. For example: -- Before edit "line1 line2" -- After edit "line1\nline2" 4. Save the monitor. Alternatively, use the following script. 1. Copy the following contents into a python script by typing: ex: /shared/tmp/http_physical_line_converter.py 2. Start the script by typing: python /shared/tmp/http_physical_line_converter.py Sample script output: # python /shared/tmp/http_physical_line_converter Running command : tmsh save sys config File : /config/partitions/part1/bigip.conf converted successfully File : /config/bigip.conf converted successfully Running command : tmsh load sys config All 2 files successfully converted ~~~~~~~~~ SCRIPT START ~~~~~~~~~~ #!/usr/bin/env python import sys import os import time import re import subprocess root = "/config/partitions" target_filename = "bigip.conf" # Get all target files in /config/partition/ def get_config_files(root, found_files=[]): if not os.path.exists(root): return [] for file_name in os.listdir(root): x = os.path.join(root, file_name) if os.path.isdir(x): get_config_files(x, found_files) else: if file_name == target_filename: found_files.append(x) return found_files # Get all ltm monitor http configurations def get_monitor_configs(data): http_monitors_list = [] temp_str = '' inside = False quotes_closed = True quotes_count = 0 for line in data.splitlines(): if line.startswith('ltm monitor http ') and line.endswith(' {') : inside = True if inside : temp_str += line + '\n' quotes_count += line.count('"') - line.count('\\"') if quotes_count % 2 : quotes_closed = False else : quotes_closed = True quotes_count = 0 if quotes_closed : if line is '}' : inside = False http_monitors_list.append(temp_str) temp_str = '' return http_monitors_list def Replace_break(InputString): return re.sub('\n', r'\\r\\n', InputString) def get_replace_string(http_monitor): _buffer_ = http_monitor # RE to grep send and recv strings from ltm http monitor configs # RE greps single line string or string inside double quotes # Returns two matched strings in tuple 1)with send 2)without send send = re.compile(r'(send (".*?(?<!\\)"|[^\n]+))', re.DOTALL) # Returns two matched strings in tuple 1)with recv 2)without recv recv = re.compile(r'(recv (".*?(?<!\\)"|[^\n]+))', re.DOTALL) _buffer_ = _buffer_.replace(re.findall(send, http_monitor)[0][1], Replace_break(re.findall(send, _buffer_)[0][1])) _buffer_ = _buffer_.replace(re.findall(recv, http_monitor)[0][1], Replace_break(re.findall(recv, _buffer_)[0][1])) return _buffer_ def modify_content(_file_, backup_files): try: buffer = None original_content = None with open(_file_ , "r") as fd: original_content = fd.read() temp_buffer = original_content backup_file = _file_ + ".backup_" + time.strftime('%Y%m%d%H%M%S') with open(backup_file , "w+") as fd: fd.write(original_content) backup_files.append(backup_file) http_monitor_formats = get_monitor_configs(temp_buffer) for each_http_monitor in http_monitor_formats: temp_buffer = temp_buffer.replace(each_http_monitor, get_replace_string(each_http_monitor)) with open(_file_, "r+") as fd: fd.write(temp_buffer) fd.truncate() return True except Exception as e: print "Error ",e," while converting file: ", _file_ print "Loading the old configurations" return False def run_command(cmd): try: print "Running command : ", cmd _cmd_out = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE, stderr=subprocess.PIPE) (_stdout_, _stderr_) = _cmd_out.communicate() if _stderr_ != '': print _stderr_ return False return True except Exception as e: print "Error: ",e return False def unlink(backup_files): for _file_ in backup_files: os.unlink(_file_) def reload_old(filenames, backup_files): for _file_, backup_file in zip (filenames, backup_files): buf = None with open(backup_file, "r") as fd: buf = fd.read() with open (_file_, "w+") as fd: fd.write(buf) fd.truncate() return run_command('tmsh save sys config') def init(): # First save all the configurations so that all config files are updated if not run_command('tmsh save sys config') : return filename_list = get_config_files(root) # Only file which is not in /config/partitions/ filename_list.append('/config/bigip.conf') number_of_files = len(filename_list) files_parsed = 0 backup_files = [] # Parse through each file and update for _file_ in filename_list: if modify_content(_file_, backup_files) : files_parsed += 1 print "File :" , _file_, " converted successfully" else: print "Unable to convert file :", _file_ break if files_parsed == number_of_files : if run_command('tmsh load sys config') : print 'All ', number_of_files, ' files successfully converted' unlink(backup_files) return else: reload_old(filename_list, backup_files) unlink(backup_files) print "Encountered an error while saving modified configs, re-loading old configurations" return run_command('tmsh save sys config') unlink(backup_files) def main(): init() if __name__ == "__main__": main() ~~~~~~~~~ SCRIPT END ~~~~~~~~~~

Fix Information

None

Behavior Change