DataDomain DD7200, DDOS 184.108.40.206 (upgrading soon)
I'm reaching out just to see what kind of dedupe you all are seeing with your Oracle Database backups, and to see if I might have an issue. We average approx 41X dedupe for all data types across the board. But, our Oracle DB backups average around 3.7X dedupe. I have spoken with our Oracle DBA's, and they are adamant that they are not running encryption or compression. They use RMAN with DDBoost to send their backups to Datadomain, so if they are not running encryption and compression, then I would thing DDBoost would be getting better rates. They are the only ones whose backups that I do not control.
I'm aware that in general, databases don't compress quite as well as other data types, but shouldn't we be getting around 15 to 20x dedupe for oracle database backups, or am I dreaming?
We are a PeopleSoft shop with Oracle 11g (11.2) as its backend. Two of my databases are getting around 44x compression and another one around 25x compression (according to mtree statistics in DD GUI). Using DDBoost as well (2.x but were same numbers in 1.x)
check filesperset setting. this should be to 1 for optimal de-duplication - anything other than 1 backups will be multiplexed which means data will be received in different order every time. compression could still be enabled and used by default even if not called to do so by the rman script. change rate could also be a factor to low de-dup.
I have asked our Oracle DBA's to look through and spot check several of their rman commands for each backup job, to see if they might be running rman compression, which will prevent ddboost compression from doing its job. Awaiting a response. Thanks for your help so far!
This may also be due to a few other factors, for example:
- Encryption of data (either in Oracle or during backup) - encrypted data does not de-duplicate well as every time a small piece of data changes the entire chunk of encrypted data appears to change (due to the encryption algorithm). When using encryption you should normally expect low de-duplication ratio but reasonable local compression ratio
- Compression of data (again either in Oracle or during backup) - as with encrypted data even if a small piece of data within the database changes the entire chunk of data which is compressed together appears to change (due to the compression algorithm). As a result compressed data normally gives poor de-duplication AND compression ratios
- Use of binary data embedded within tables - binary data such as image/media files tends to be pre-compressed by its codec so is essentially compressed data and all of the points above apply
- Backing up archive logs - archive logs contain transactional data which tends to be very unique from backup to backup - as a result de-duplication ratio for archive log backups is generally very poor however these should still compress (if not compressed during the backup)
- Very high change rate within your databases - if the majority of the data changes between backups then most of each backup will appear to be 'new' data which the DDR has not seen before. As a result it does not de-duplicate well (however should still compress if not compressed during the backup)
Hopefully that helps a little.
Ok, I have been working with our Oracle DBA's, and looking over things.
Here is what they sent me.
CONFIGURE ENCRYPTION FOR DATABASE OFF
CONFIGURE COMPRESSION ALGORITHM 'BASIC' (so I started digging into this setting, because I know there is also a 'NONE' option, and I brought that up to the DBA. He said that 'BASIC' is the default setting, but that on the actual backup piece, the setting is
64435 Incr 0 14.41G SBT_TAPE 00:01:09 16-NOV-15
BP Key: 64435 Status: AVAILABLE Compressed: NO Tag:
He states that encryption is also not being used.
So, I ran ddboost storage-unit show compression against the storage unit that rman writes to, and looking through, even the databases that just arent used much anymore are still only showing 1.9 to 3.5x, so it doesnt appear that archival log busyness is the issue.
This is really odd to me. I mean, perhaps 3.5x would be normal for us, but so far, I have yet to see where anyone else is sending Oracle database backups to datadomain and getting compression this low.
DD9800 DDOS 220.127.116.11 NBU 7.7.2 environment Linux/Windows OS
Happy to find that this is an active thread, I have a very similar situation with RMAN backups to DD that are getting very poor de-duplication/compression and eats up all our storage on the DD. Going to ask the DBA's for further info.
Here is what I can see from looking at the /usr/openv/netbackup/logs/backint/log.%date
Backup starts and I can see some parameters there, my question is, should the FILESPERSET = 1 be shown here ?
INF - Starting bpbrm
INF - Multiplexing = 0
INF - Compression = 0
INF - Encrypt = 0
Sorry for my English, I'm writing from Poland.
According to chart which You included, it seems that Oracle database has one full backup per week and every other day incremental backups. Because of incremental backups You shoudn't expect deduplication ratio above 10. Main advantage of Oracle backup with Datadomain is that You don't have full backup penalty. You should try to do full backup every day and deduplication ratio will be above 15. It's even possible that incremental backups take more space then everyday full backups.
No worries about your English. It is much better than many people born in the U.S. :-)
Yes, I have taken that into consideration, that deduplication would be lower if only incremental backups happen daily. Thats mainly unique data, which cannot deduplicate, That would explain why my daily incrementals would be 1 to 1.5. 1x compression is no compression at all. But, it seems that my full backups should be much higher than 7x average deduplication. Granted, I do not know what percentage of those full backups are new data. If its a high percentage, the dedupe would be low.