Last Modified: Jul 12, 2023
Affected Product(s):
BIG-IQ Config Mgmt Infrastructure
Opened: Oct 17, 2019 Severity: 3-Major
Not all existing snapshots appear in UI if an error occurs during snapshot removal.
ES snapshots take up space in the /var filesystem which can fill up and cause problems with other systems that need available space.
When you select multiple ES snapshots for deletion sometimes only one is removed from the filesystem while the others remain but no longer show up in the UI.
To work around this in the UI, select only a single ES snapshot at a time for deletion. You can also delete the snapshots from the command line as follows: 1. curl -s localhost:9200/_snapshot/backup/_all | jq > /tmp/es_snapshot_summary 2. From /tmp/es_snapshot_summary get all snapshot names and save in /tmp/snapshot_names 3. Iterate /tmp/snapshot_names to fetch snapshot name one at a time and search for the presence in restcurl -u user:password /mgmt/cm/shared/esmgmt/es-snapshots/ 4. If snapshot name is present in /mgmt/cm/shared/esmgmt/es-snapshots/ then it is not a stale entry and nothing to be done and continue to iterate. 5. If snapshot name is not present in /mgmt/cm/shared/esmgmt/es-snapshots/ then it is a stale entry and trigger { curl -X DELETE "localhost:9200/_snapshot/backup/<snapshot_name>?pretty" } for cleanup. If delete operation failed and reason for failure is 503[another delete request in progress] then wait for sometime [30 seconds] and reattempt[max reattempt count 3] and proceed.
None