Last Modified: Dec 27, 2018
See more info
Known Affected Versions:
12.1.0, 12.1.0 HF1, 12.1.0 HF2, 12.1.1, 12.1.1 HF1, 12.1.1 HF2, 12.1.2, 12.1.2 HF1, 12.1.2 HF2, 12.1.3, 18.104.22.168, 22.214.171.124, 126.96.36.199, 188.8.131.52, 184.108.40.206, 220.127.116.11, 18.104.22.168, 12.1.4, 13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1
Opened: Apr 17, 2017
Related AskF5 Article: K00415052
The BMC (AOM) firmware intermittently gets into a state where the BMC log-cleaning services stops working and causes the logging subsystem to grow and use all available memory. When in this state, if a new version of BIG-IP software is installed, it attempts to update the BMC firmware, which fails because there is not enough memory available to support firmware upgrade. As a result of this issue, you may encounter one or more of the following symptoms: -- You see the following output on the serial console: Retrigger udev events[ OK ] Starting sshd: [ OK ] Checking for firmware updates: Updating AOM to /usr/firmware/bmc_shuttle_2.51.ima_enc Updating AOM ... DO NOT POWER DOWN ... THIS MAY TAKE UP TO 12 MINUTES TO COMPLETE Updating AOM ... DO NOT POWER DOWN ... THIS MAY TAKE UP TO 12 MINUTES TO COMPLETE (elapsed seconds:1) -- The unit does not boot up to the host. -- The unit reports firmware upgrade in progress on the LCD. -- The Configuration utility is unavailable.
Once BMC upgrade attempt fails, BMC gets stuck in firmware update mode, which causes failure of all subsequent IPMI commands from the BIG-IP system. In addition, the BIG-IP system cannot read the inventory information, which causes license check failure.
This issue occurs when all of the following conditions are met: -- You have one of the following BIG-IP platforms: * i2x00 * i4x00 * i5x00 * i7x00 * i10x00 * HRC-i2x00 * HRC-i5x00 * HRC-i10x00 -- You perform an upgrade from any of the following BIG-IP versions: * 13.0.0 * 12.1.2 * 12.1.1 -- You have had the BIG-IP system powered on for approximately five weeks or more.
Before new BIG-IP software is activated, reset BMC using one of these methods: -- Invoke AOM menu from system console port (ESC-Shift-9) and select A ( AOM Reset). -- Power cycle the system. -- From the BIG-IP command line, run the following command: ipmiutil reset -k. Note: If, after trying the above workarounds first, the problem still persists, then, you might need to power down the device and physically remove power for a few seconds. In most cases the above workaround will work, but if it does not then a power cycle may be necessary.