PPDM Oracle/ SQL: The /opt directory ran out of space for Oracle and SQL backups
Summary: A folder under /opt named /v1/backups. This is taking up 15 GB of space.
Symptoms
We have noted that there is a folder under /opt named /v1/backups. This is taking up 15 GB of space.
Cause
The /v1/backups directory is created by the Oracle/SQL plugin / agent.
The data in the /backups directory is from four SQLite databases that are created every hour.
- Copies.db - This contains the backups' naturalID - which is the link between the copies.db and the elastic search database on the PPDM appliance.
- Sessions.db
- This covers information about the backups - uses sessionid notation.
- Deletions - uses what look like UUIDs or task notation
- Objects.db
- Resources.db
- Shows addon information
- Shows assets
Every hour, we take a copy of the four SQLite databases mentioned above and store them in a timestamped directory in /v1/backups.
E.g.:
@DX1907 backups]$ ls -lastotal 68820 drwxrwxrwx. 168 root root 20480 Mar 3 15:14 .4 drwxrwxrwx. 3 root root 4096 Mar 3 12:58 ..4 drwxrwxrwx. 2 root root 4096 Feb 24 16:11 2021-02-24_16-11-28_1614168688.5211214 drwxrwxrwx. 2 root root 4096 Feb 24 17:11 2021-02-24_17-11-29_1614172289.2606364 drwxrwxrwx. 2 root root 4096 Feb 24 18:11 2021-02-24_18-11-30_1614175890.0498684 drwxrwxrwx. 2 root root 4096 Feb 24 19:11 2021-02-24_19-11-30_1614179490.990923 |
We can see directories from Feb 24th at 1600, 1700, 1800 and 1900 hours.
By default, the Oracle agent retains the backups for 7 days (168 hours).
So if you look at the impact of 168 iterations of the SQLite DBs:
drwxrwxrwx. 168 root root 20480 Mar 3 15:14 backups-rw-r--r--. 1 root root 80535552 Mar 3 12:33 copies.db-rw-r--r--. 1 root root 12288 Mar 3 12:48 objects.db-rw-r--r--. 1 root root 40960 Mar 3 12:58 resources.db-rw-r--r--. 1 root root 376832 Mar 3 04:04 sessions.db |
The objects, resources and sessions databases are trivial in size.
But the copies.db is about 80 MB. So take that and multiply it by 168 and you get about 13.5 GB, that's where the size problem comes from.
Resolution
We need to modify the backup retention settings for the Oracle agent.
This can be done by modifying the config.yml file.
In the config.yml, you would see
backup_retention:# agent service backup retention time in hours, default is 7days = 168h.168We would want to change the values tobackup_retention:# agent service backup retention time in hours, default is 7days = 168h.48 |
Then we would want to restart the agentsvc.
It may take five to ten minutes for the changes to take effect.