Low deduplication for Oracle DB backup

Todd,

We are a PeopleSoft shop with Oracle 11g (11.2) as its backend.  Two of my databases are getting around 44x compression and another one around 25x compression (according to mtree statistics in DD GUI). Using DDBoost as well (2.x but were same numbers in 1.x)

check filesperset setting. this should be to 1 for optimal de-duplication - anything other than 1 backups will be multiplexed which means data will be received in different order every time.  compression could still be enabled and used by default even if not called to do so by the rman script. change rate could also be a factor to low de-dup.

cheers,

-e

I have asked our Oracle DBA's to look through and spot check several of their rman commands for each backup job, to see if they might be running rman compression, which will prevent ddboost compression from doing its job. Awaiting a response. Thanks for your help so far!

This may also be due to a few other factors, for example:

- Encryption of data (either in Oracle or during backup) - encrypted data does not de-duplicate well as every time a small piece of data changes the entire chunk of encrypted data appears to change (due to the encryption algorithm). When using encryption you should normally expect low de-duplication ratio but reasonable local compression ratio

- Compression of data (again either in Oracle or during backup) - as with encrypted data even if a small piece of data within the database changes the entire chunk of data which is compressed together appears to change (due to the compression algorithm). As a result compressed data normally gives poor de-duplication AND compression ratios

- Use of binary data embedded within tables - binary data such as image/media files tends to be pre-compressed by its codec so is essentially compressed data and all of the points above apply

- Backing up archive logs - archive logs contain transactional data which tends to be very unique from backup to backup - as a result de-duplication ratio for archive log backups is generally very poor however these should still compress (if not compressed during the backup)

- Very high change rate within your databases - if the majority of the data changes between backups then most of each backup will appear to be 'new' data which the DDR has not seen before. As a result it does not de-duplicate well (however should still compress if not compressed during the backup)

Hopefully that helps a little.

Thanks, James

Ok,  I have been working with our Oracle DBA's, and looking over things.

Here is what they sent me.

CONFIGURE ENCRYPTION FOR DATABASE OFF

CONFIGURE COMPRESSION ALGORITHM 'BASIC'  (so I started digging into this setting, because I know there is also a 'NONE' option, and I brought that up to the DBA. He said that 'BASIC' is the default setting, but that on the actual backup piece, the setting is

64435   Incr 0 14.41G     SBT_TAPE 00:01:09     16-NOV-15

          BP Key: 64435 Status: AVAILABLE  Compressed: NO  Tag:

He states that encryption is also not being used.

So, I ran ddboost storage-unit show compression against the storage unit that rman writes to, and looking through, even the databases that just arent used much anymore are still only showing 1.9 to 3.5x, so it doesnt appear that archival log busyness is the issue.

This is really odd to me. I mean, perhaps 3.5x would be normal for us, but so far, I have yet to see where anyone else is sending Oracle database backups to datadomain and getting compression this low.

DD9800 DDOS 5.6.2.0 NBU 7.7.2 environment Linux/Windows OS

Happy to find that this is an active thread, I have a very similar situation with RMAN backups to DD that are getting very poor de-duplication/compression and eats up all our storage on the DD.   Going to ask the DBA's for further info.

Here is what I can see from looking at the /usr/openv/netbackup/logs/backint/log.%date

Backup starts and I can see some parameters there, my question is, should the FILESPERSET = 1 be shown here ?

==========

Initiating backup

INF - Starting bpbrm

.....

INF - Multiplexing = 0

INF - Compression = 0

INF - Encrypt = 0

Hello,

Sorry for my English, I'm writing from Poland.

According to chart which You included, it seems that Oracle database has one full backup per week and every other day incremental backups. Because of incremental backups You shoudn't expect deduplication ratio above 10. Main advantage of Oracle backup with Datadomain is that You don't have full backup penalty. You should try to do full backup every day and deduplication ratio will be above 15. It's even possible that incremental backups take more space then everyday full backups.

Best regards,

Konrad

Hello Radeqs,

No worries about your English. It is much better than many people born in the U.S. :-)

Yes, I have taken that into consideration, that deduplication would be lower if only incremental backups happen daily. Thats mainly unique data, which cannot deduplicate, That would explain why my daily incrementals would be 1 to 1.5. 1x compression is no compression at all. But, it seems that my full backups should be much higher than 7x average deduplication. Granted, I do not know what percentage of those full backups are new data. If its a high percentage, the dedupe would be low.

Hello again .

I'm not sure, but I think that deduplication ratio for your incremental backups is about 3 (blue to red ratio on the chart). Maybe incremental backups (with block change tracking) is a must, because of size of your database. But if there is a chance to do a full backup in a working day backup window, maybe You should try full backups for couple days without incrementals.

BTW. What is the reason that full backups have so big diffrence in size before deduplication? Maybe your DB really has huge changes week to week?