Bug ID 656828: Setting the master key after upgrading a system with a large configuration from 5.x to 5.2 could result in an unsuccessful encryption of objects.

Last Modified: Oct 01, 2018

Bug Tracker

Affected Product:  See more info
BIG-IQ Platform(all modules)

Known Affected Versions:
5.1.0

Fixed In:
5.4.0, 5.3.0

Opened: Apr 07, 2017
Severity: 2-Critical

Symptoms

After upgrading the BIG-IQ system from 5.x to 5.2, when the user logs into the BIG-IQ UI, the user will be required to go through the setup wizard. When the master key passphrase is entered and the Next button clicked, the master key is created and the encryption upgrade starts. The following two symptoms can occur: Symptom 1: If the encryption upgrade does not finish within five minutes, the user will see a 504 gateway timeout exception in the UI. This is a possible indication that the encryption upgrade will not succeed, so the user should click the Dismiss button, log out from the UI, and check to see if symptom 2 occurs after waiting another five minutes. Symptom 2: If the encryption upgrade does not complete in ten minutes, in the /var/log/restjavad.0.log file the following error message is observed: [ERROR][12 Apr 2017 11:07:00 EDT][/cm/shared/secure-storage/masterkey SecureStorageMasterkeyGenerator] The BIG-IQ ran into error 'Encryption upgrade has failed to run to completion due to Timed out during execution of command. This may result in some attributes that are encrypted with the old encryption scheme that need to be manually upgraded.' when upgrading encrypted values. This may cause some encrypted values to be unusable. If Symptoms 1 and 2 are both seen, the customer should proceed with the workaround.

Impact

If the encryption upgrade fails, the upgraded BIG-IQ system will be unstable to use. There will be several errors in the product and in the log files.

Conditions

The pre-upgraded 5.x system has large number of objects requiring encryption. example: The BIG-IQ system managed several hundred BIG-IP's, had several hundred rules, etc (a very large system) then such a system upon upgrade to 5.2 could have an issue setting the encryption master key upon first logging in to the BIG-IQ 5.2 UI.

Workaround

If both symptoms 1 and 2 are seen, the customer can work around the issue as follows: 1. Log in to the BIG-IQ shell (not the UI) 2. cd /var/config/rest/tokuupgrade/encryption 3. sh run_encryption_upgrade.sh 4. Wait for the execution of this command to complete. When the execution completes, the following message will be displayed: "The Encryption upgrade script is complete" 5. Log back in to the UI and finish executing the setup wizard.

Fix Information

None

Behavior Change