Bug ID 739507: How to recover from a failed state due to FIPS integrity check

Last Modified: Apr 10, 2019

Bug Tracker

Affected Product:  See more info
BIG-IP TMOS(all modules)

Known Affected Versions:
13.0.0, 13.0.0 HF1, 13.0.0 HF2, 13.0.0 HF3, 13.0.1, 13.1.0, 13.1.0.1, 13.1.0.2, 13.1.0.3, 13.1.0.4, 13.1.0.5, 13.1.0.6, 13.1.0.7, 13.1.0.8, 13.1.1, 13.1.1.1

Fixed In:
13.1.1.2

Opened: Aug 07, 2018
Severity: 2-Critical

Symptoms

After FIPS 140-2 license is installed on FIPS-certified hardware devices, and the device rebooted, the system halts upon performing FIPS integrity check.

Impact

The device is halted and cannot be used.

Conditions

[1] Some system applications, monitored by FIPS 140-2, get routinely changed. [2] The device was containing a FIPS 140-2 enabled license installed. [3] The device operator installs a FIPS 140-2 enabled license [4] The device is rebooted

Workaround

Workaround: [1] The device needs to have serial console access (Telnet). [2] From the Telnet console, enter the GRUB menu and boot into a different partition not having a FIPS 140-2 enabled license. [3] Examine the contents of file /config/fipserr which will show the files that were changed, leading to failure of the FIPS 140-2 license-enabled partition. [4] Restore those files to their original ones and reboot. If system still halts, repeat from Step [1] above, until this no longer happens.

Fix Information

Here are the steps, in summary form. [1] Connect a terminal to the BIG-IP serial console port [2] From the Telnet console, enter the GRUB menu. [3] Before the countdown expires, use the Up Arrow and Down Arrow keys to stop the countdown, and select the appropriate boot image. [4] Press the key 'E' to start the edit options. A new GRUB menu displays. [5] Use the Up Arrow and Down Arrow keys to navigate to the line that contains the keyword "module". [6] Add a space, followed by NO_FIPS_INTEGRITY=1. DO NOT press ENTER. [7] Press the Ctrl-X sequence or the F10 key to restart the system using the modified options. The machine will boot into the partition containing FIPS 140-2-enabled license. [8] Examine the content of file /config/f5_public/fipserr to ascertain the cause of the FIPS module startup error. [9] Fix the problem reported in the aforementioned error file. [10] Run the test tool /usr/libexec/sys-eicheck.py to ensure that no fatal error is reported, such as: Integrity Check Result: [ FAIL ] If fatal error persists, DO NOT REBOOT (otherwise the system will go into the halt state, and the steps starting from Step [1] will need to be repeated). Instead, fix the problematic files reported. Re-run the test tool until no error is seen.

Behavior Change