RMAN backup is failing in legato backup server. Please help on this

[sushin]

Hello Team,

In my setup one of the database RMAN backup is failing .. The error details mentioning below. the file systems got umounted yesterday due to some power issue . the client is solaris , But now all the file systems are proper and it is working fine.

Bellow is the detail list of all events that took place on our server

=== ODM Question ===
Deleted archivelogs from a  location that was  full. Performed a "crosscheck copy of archivelog all" to  normalize  Archivelog sequences. Then run an RMAN backup which yielded these   errors... .Please help us on this.

RMAN> backup database
2> ;

Starting backup at  06-DEC-10
using channel ORA_SBT_TAPE_1
using channel  ORA_SBT_TAPE_2

channel ORA_SBT_TAPE_1: starting full datafile backup  set
channel ORA_SBT_TAPE_1: specifying datafile(s) in backup set
input  datafile file number=00006  name=/rept/oradata/u02/PRODREPT/REPTT01_1.dbf
channel ORA_SBT_TAPE_1:  starting piece 1 at 06-DEC-10
channel ORA_SBT_TAPE_2: starting full datafile  backup set
channel ORA_SBT_TAPE_2: specifying datafile(s) in backup  set
input datafile file number=00007  name=/rept/oradata/u02/PRODREPT/REPTT02_1.dbf
channel ORA_SBT_TAPE_2:  starting piece 1 at 06-DEC-10
RMAN-03009: failure of backup command on  ORA_SBT_TAPE_1 channel at 12/06/2010 16:39:57
ORA-19506: failed to create  sequential file,, parms=""
ORA-27028: skgfqcre: sbtbackup  returned error
ORA-19511: Error received from media manager layer, error  text:
lnm_index_cfx_connect retry failed. See privious error messages.  (3:3:11)

Verified channels via RMAN

RMAN> show  channel;

RMAN configuration parameters for database with db_unique_name  PRODREPT are:
CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' PARMS  'ENV=(NSR_SERVER=tzdarhq-bksrv,NSR_DATA_VOLUME_POOL=REPT,NSR_CLIENT=rept-lh

ABBA.MTENGA@TZ.AIRTEL.COM - December 7, 2010 12:13:23 PM  GMT+03:00 [Customer Problem Description]

ABBA.MTENGA@TZ.AIRTEL.COM - December 7, 2010 12:13:22 PM  GMT+03:00 [Customer Problem Description]

Problem Description: Deleted archivelogs from a location  that was  full. Performed a "crosscheck copy of archivelog all" to normalize   Archivelog sequences. Then run an RMAN backup which yielded these   errors... RMAN> backup database 2> ; Starting backup at  06-DEC-10 using channel ORA_SBT_TAPE_1 using channel  ORA_SBT_TAPE_2 channel ORA_SBT_TAPE_1: starting full datafile backup  set channel ORA_SBT_TAPE_1: specifying datafile(s) in backup set input  datafile file number=00006  name=/rept/oradata/u02/PRODREPT/REPTT01_1.dbf channel ORA_SBT_TAPE_1:  starting piece 1 at 06-DEC-10 channel ORA_SBT_TAPE_2: starting full datafile  backup set channel ORA_SBT_TAPE_2: specifying datafile(s) in backup  set input datafile file number=00007  name=/rept/oradata/u02/PRODREPT/REPTT02_1.dbf channel ORA_SBT_TAPE_2:  starting piece 1 at 06-DEC-10 RMAN-03009: failure of backup command on  ORA_SBT_TAPE_1 channel at 12/06/2010 16:39:57 ORA-19506: failed to create  sequential file,, parms="" ORA-27028: skgfqcre: sbtbackup  returned error ORA-19511: Error received from media manager layer, error  text: lnm_index_cfx_connect retry failed. See privious error messages.  (3:3:11) Verified channels via RMAN RMAN> show  channel; RMAN configuration parameters for database with db_unique_name  PRODREPT are: CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' PARMS  'ENV=(NSR_SERVER=tzdarhq-bksrv,NSR_DATA_VOLUME_POOL=REPT,NSR_CLIENT=rept-lh

[masonb]

Sushin,

Can you please check the following: -

1. Check name resolution for server and client host names on all referred to in the hosts files and in the RMAN script.

2. Set all the ulimit values on the system to unlimited.

3. Restart the listener and re-run the backup.

Regards,

Bill Mason

[sushin]

Hello William Mason ,

Thank you for the quick reply..  I will try and update you...

once againe thanks

With regards

sushin.k

[vsampy]

Hi Sushin

This could be due to the power issue, the backups failed and has left some remnants, also check the connectivity between the Oracle Client and the Backup Server.

- Check the Oracle Client, see that there is no process of previous RMAN failure.

- Check the connectivity between the Networker backup server and the Oracle Client and vice versa

- on the client stop the networker services

- rename /nsr/tmp/ folder to /nsr/tmp_old

- restart the services

- do the same on the backup server

- in addition on the backup server rename /nsr/res/jobsdb/ folder to /nsr/res/jobsdb_old

if the problem persist, initiate debugging by adding NSR_DEBUG_FILE=/nsr/applogs/debug.log and NSR_DEBUG_LEVEL=9, to the ENV

example: 'ENV=(NSR_SERVER=tzdarhq-bksrv,NSR_DEBUG_FILE=  ,NSR_DEBUG_LEVEL=9,NSR_DATA_VOLUME_POOL=REPT,NSR_CLIENT=rept-lh

Let us know how things proceed.