ScotN
2 Iron

Oracle Hangs running rman to share.

Hi, 

   We have a Linux Dell R610 SAN attached that I am trying to convert to a DD670 backup.  Conventionally our Linux hosts backup to NFS.

When rman tries to write to the data domain oracle and the system locks up to the point it needs to be hard reset.

Has anyone had experience in this area ? I suspect the backup is just saturating Linux host resources. All other Linux and Windows systems are not having issues writing to the data domain and  DataDomain resources are well within tolerance. 

rman cmdfile

run {

crosscheck archivelog all;

delete noprompt obsolete device type DISK;

delete noprompt backup completed before 'sysdate';

delete noprompt expired archivelog all;

allocate channel ch1 device type DISK format '/nfs/backup/oralce_%d_%U';

allocate channel ch2 device type DISK format '/nfs/backup/oralce_%d_%U';

allocate channel ch3 device type DISK format '/nfs/backup/oralce_%d_%U';

allocate channel ch4 device type DISK format '/nfs/backup/oralce_%d_%U';

allocate channel ch5 device type DISK format '/nfs/backup/oralce_%d_%U';

allocate channel ch6 device type DISK format '/nfs/backup/oralce_%d_%U';

sql 'alter system archive log current';

backup current controlfile format '/nfs/backup/oralce_cntrl_%d_%U';

backup database plus archivelog not backed up delete input tag='SUNDAY_FULL BACKUP';

crosscheck backup;

crosscheck backupset;

crosscheck copy;

release channel ch1;

release channel ch2;

release channel ch3;

release channel ch4;

release channel ch5;

release channel ch6;

}

Tags (2)
0 Kudos
8 Replies
ble1
5 Iridium

Re: Oracle Hangs running rman to share.

If you suspect saturation, try with single channel allocated.  You can also set debug mode to get more idea of what is going and where (and you can check it oracle where the hang is too by checking process properties and seeing where the wait comes from).

0 Kudos
ScotN
2 Iron

Re: Oracle Hangs running rman to share.

I suspect saturation but it's a bit odd. Network goes to a few kb and system become unusable.

This may simply be a DBA overrunning his system by letting it consume the host resources.  But because it hangs writing to DD and not to a backup SAN lun its obviously my problem Forget that there are no other MsSQL or Mysql backup issues even during the same timeframe. 

SAR reports do show almost every cpu core going >20% I/O wait. But I do not see equal saturation on Disk, swap or any other resource in the same SAR report.

Using Ksar to evaluate data. 

Posted another question about ddboost license terms as well.  Just can't find a smoking gun.  

0 Kudos
nithin_s
1 Nickel

Re: Oracle Hangs running rman to share.

Are your LINUX/WINDOWs systems which are working fine from a backup software writing to Data Domain? Are they DDBOOST aware?

If Oracle writing to DD NFS, performance is purely on your network.

You can think of using DDBOOST for RMAN API/plugin. It might have license associated within but compression/dedup rate is huge difference compared to NFS and you would save on Storage cost making it less difference. Also, it's a great UI and easily managable with much better performance on backup/restore time.

http://www.emc.com/collateral/hardware/white-papers/h10683-dd-boost-oracle-rman-tech-review-wp.pdf

dynamox
6 Gallium

Re: Oracle Hangs running rman to share.

Nithin S wrote:

You can think of using DDBOOST for RMAN API/plugin. It might have license associated within but compression/dedup rate is huge difference compared to NFS and you would save on Storage cost making it less difference. Also, it's a great UI and easily managable with much better performance on backup/restore time.

http://www.emc.com/collateral/hardware/white-papers/h10683-dd-boost-oracle-rman-tech-review-wp.pdf

How's using DDBoost versus NFS will save on storage cost , considering i am writing to the same DD ?  Performance sure, but cost ?

0 Kudos
nithin_s
1 Nickel

Re: Oracle Hangs running rman to share.

Well, DDBOOST architecture has better compression/dedup rates compared to traditional NFS. If you have a DDBOOST plugin for RMAN, it does client side deduplication and very less amount of data is sent over LAN/WAN from the client along with great dedup/comp ratio.

0 Kudos
dynamox
6 Gallium

Re: Oracle Hangs running rman to share.

from data domain perspective it does not matter how the data got there, NFS or DDBoost.  It will provide the same dedupe/compression regardless of the protocol.

0 Kudos
Highlighted
PS-VCORE
1 Copper

Re: Oracle Hangs running rman to share.

However.... If you enable CBT (Change Block Tracking) on Oracle, AND use DDBoost for RMAN, your backup windows shrink tremendously, as only the changed blocks are backed up each time, and pointers are used for the rest. Effectively a synthetic full.

0 Kudos
dynamox
6 Gallium

Re: Oracle Hangs running rman to share.

once again, there are absolutely no savings on back-end storage ..regardless of how the data got there. DDBoost, VTL, CIFS, NFS.