Bug ID 901625: Analytics dashboards after upgrading to BIG-IQ version 7.1

Last Modified: Oct 14, 2020

Bug Tracker

Affected Product:  See more info
BIG-IQ AppIQ(all modules)

Known Affected Versions:
7.1.0

Fixed In:
7.1.0.1

Opened: Apr 21, 2020
Severity: 3-Major

Symptoms

In rare instances, statistics data previously gathered might not display in the Applications > Monitoring > DASHBOARDS (excluding Access reports) Application Service dashboards after upgrading to BIG-IQ version 7.1.

Impact

Statistics data gathered before upgrading to BIG-IQ v7.1 might not display.

Conditions

1. Upgrade any supported version of BIG-IQ with DCDs to version 7.1. 2. From the user interface, analytics do not include data prior to the upgrade. 3. Two to four days after a sucessful upgrade, the the BIG-IQ logs And the /var/log/restjavad.0.log (or /var/log/restjavad.1.log if the log was rotated) must contain the following: "Analytics script /var/config/appiq/upgrade/upgrade_analytics_reindex_tmstat_statistics_7_0_to_7_1.py finish successfully" 4. You see (3.) but the upgrade scripts are still unable to complete processing data because the maximum retries was reached. When this happen, BIG-IQ logs the following:/var/log/upgrade_analytics_data.log at this case it must contain ERROR log (not WARN) with the following text: "ES Post failed for url" 5. From the BIG-IQ command line, run the following command to see if there is some old analytics data that was not converted: curl localhost:9200/_cat/indices/statistics* | grep -v '\-group' If this command returns some indexes after the process of post-upgrade finish, it meanst not all old analytics data was upgraded and do not appear after an upgrade.

Workaround

After you have verified that all 5 conditions are met the workaround is as follows: 1. Copy the pre-upgrade script to a file /tmp/script.sh For pre-upgrade procedures, see https://techdocs.f5.com/en-us/bigiq-7-1-0/upgrading-big-iq-5-4-7-0-dcd-cluster-latest/upgrading-bigiq-version-5-4-7-0_overview.html 2. Run the file with the added script: source /tmp/script.sh 3. Run: python2.7 /var/config/appiq/upgrade/upgrade_analytics_reindex_tmstat_statistics_7_0_to_7_1.py 4. The script processes unfinished old analytics data. You can monitor the progress at the /var/log/upgrade_analytics_data.log At the end of the process you should see at the log: REINDEX statistics completed successfully The script might run from couple of hours to couple or days depending on the data size left.

Fix Information

Statistics gathered prior to the upgrade now appear in the dashboards as expected after upgrading.

Behavior Change