tolejniczak
1 Nickel

Recovering rman data from Networker

Jump to solution

Hello,

I need to recover dropped partition from rman backup (Oracle 10). The last level 0 backup that contains this partition was performed on 09/27/15. I'm using EMC networker for bakcup.

There was no catalog database and now backups from 09/27/15 are no longer visible in RMAN, because control_file_record_keep_time was set too low.

However some of those backups are still on tape - they are visible in mminfo results:

/usr/sbin/nsr/mminfo -s bdlbkp -o t -r "ssid,name,totalsize,savetime,volume,state" -q "client=sdbkp"

...

5342625141 RMAN:/home/scripts/sddb_backup.rcv 327 GB 09/27/15 SD72LT E

5264665370 RMAN:/home/scripts/sddb_backup.rcv 122 GB 09/27/15 SD72LT E

5315742731 RMAN:/home/scripts/sddb_backup.rcv 124 GB 09/27/15 SD72LT E

5512210329 RMAN:/home/scripts/sddb_backup.rcv 115 GB 09/27/15 SD72LT E

...

The saveset 5342625141 corresponds to backupiece containing tablespace with dropped partition (results of "list backup" recorderd at the time those backups were still visible in RMAN - now RMAN does not see them):

BS Key  Type LV Size       Device Type Elapsed Time Completion Time

------- ---- -- ---------- ----------- ------------ ---------------

124134  Incr 0  312.59G    SBT_TAPE    01:39:23     27-SEP-15     

        BP Key: 182503   Status: AVAILABLE  Compressed: NO  Tag: TAG20150927T0

        Handle: db_SDDB_214513_948961110   Media:

  List of Datafiles in backup set 121545

  File LV Type Ckp SCN    Ckp Time  Name

  ---- -- ---- ---------- --------- ----

...

When I tried to recover backup directly by SSID from tape to disk (in order to register copy on disk in RMAN catalog) I got the following error:

# /usr/sbin/nsr/recover -s bdlbkp -S 5242625441 -d /tmp

Total estimated disk space needed for recover is 327 GB.

Recovering 1 files within RMAN:\home\scripts\ into /tmp

Permission denied by server bdlbkp : Cross platform recovery not supported

Both backup server and destination machine have Solaris.

Then I tried to register backup in RMAN:

RMAN> catalog device type 'SBT_TAPE' backuppiece 'db_SDDB_214513_948961110';

I got the following error:

allocated channel: ORA_SBT_TAPE_1

channel ORA_SBT_TAPE_1: sid=163 devtype=SBT_TAPE

channel ORA_SBT_TAPE_1: NMO v4.2.0.0

allocated channel: ORA_SBT_TAPE_2

channel ORA_SBT_TAPE_2: sid=47 devtype=SBT_TAPE

channel ORA_SBT_TAPE_2: NMO v4.2.0.0

allocated channel: ORA_SBT_TAPE_3

channel ORA_SBT_TAPE_3: sid=45 devtype=SBT_TAPE

channel ORA_SBT_TAPE_3: NMO v4.2.0.0

allocated channel: ORA_SBT_TAPE_4

channel ORA_SBT_TAPE_4: sid=52 devtype=SBT_TAPE

channel ORA_SBT_TAPE_4: NMO v4.2.0.0

allocated channel: ORA_SBT_TAPE_5

channel ORA_SBT_TAPE_5: sid=182 devtype=SBT_TAPE

channel ORA_SBT_TAPE_5: NMO v4.2.0.0

allocated channel: ORA_SBT_TAPE_6

channel ORA_SBT_TAPE_6: sid=183 devtype=SBT_TAPE

channel ORA_SBT_TAPE_6: NMO v4.2.0.0

ORA-19870: error reading backup piece db_SDDB_214513_948961110

ORA-19507: failed to retrieve sequential file, handle="db_SDDB_214513_948961110", parms=""

ORA-27029: skgfrtrv: sbtrestore returned error

ORA-19511: Error received from media manager layer, error text:

   Could not locate the LNM save file 'db_SDDB_214513_948961110' on server 'bdlbkp

'. (2:9:0)

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of catalog command at 01/20/2016 13:35:06

RMAN-06209: List of failed objects

RMAN-06211: ==========================

RMAN-06212:   Object Type   Filename/Handle

RMAN-06213: --------------- ---------------------------------------------------

RMAN-06214: Backup Piece    db_SDDB_214513_948961110

Another approach I tried was restoring older controlfile, that still contained records of those backups.

When I tried to restore controlfile that still contained records of backups I got:

run {

set until time "to_date('2015-11-27 00:00:00','yyyy-mm-dd hh24:mi:ss')";

allocate channel c0_db device type sbt

parms="ENV=(NSR_SERVER=bdlbkp, NSR_CLIENT=sdbkp)";

allocate channel c1_db device type sbt

parms="ENV=(NSR_SERVER=bdlbkp, NSR_CLIENT=sdbkp)";

restore controlfile to '/tmp/cfile';

}

executing command: SET until clause

allocated channel: c0_db

channel c0_db: sid=130 devtype=SBT_TAPE

channel c0_db: NMO v4.2.0.0

allocated channel: c1_db

channel c1_db: sid=55 devtype=SBT_TAPE

channel c1_db: NMO v4.2.0.0

Starting restore at 20-JAN-16

channel c0_db: starting datafile backupset restore

channel c0_db: restoring control file

output filename=/tmp/cfile

channel c0_db: reading from backup piece c-2315614121-20151102-02

ORA-19870: error reading backup piece c-2315614121-20151102-02

ORA-19507: failed to retrieve sequential file, handle="c-2315614121-20151102-02", parms=""

ORA-27029: skgfrtrv: sbtrestore returned error

ORA-19511: Error received from media manager layer, error text:

   Could not locate the LNM save file 'c-2315614121-20151102-02' on server 'bdlbkp

'. (2:9:0)

failover to previous backup

channel c0_db: starting datafile backupset restore

channel c0_db: restoring control file

output filename=/tmp/cfile

channel c0_db: reading from backup piece c-2315614121-20151102-01

ORA-19870: error reading backup piece c-2315614121-20151102-01

ORA-19507: failed to retrieve sequential file, handle="c-2315614121-20151102-01", parms=""

ORA-27029: skgfrtrv: sbtrestore returned error

ORA-19511: Error received from media manager layer, error text:

   A backup or restore session is already in progress. (100:17:0)

failover to previous backup

... // ("a backup or restore session is already in progress" error for all previous backups)

released channel: c0_db

released channel: c1_db

RMAN-00571: ===========================================================

RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============

RMAN-00571: ===========================================================

RMAN-03002: failure of restore command at 01/20/2016 14:45:10

RMAN-06026: some targets not found - aborting restore

RMAN-06101: no channel to restore a backup or copy of the control file

Even though controlfile backup corresponding to "c-2315614121-20151102-02" is visible in mminfo results:

3463255223 RMAN:/home/scripts/sddb_backup_lvl1.rcv 20 MB 11/02/15 SD61LT

Mminfo shows however that there are no savesets on tape for 03/11/15-11/12/15, even though they are visible in RMAN "list backup". Maybe that is the reason?

Restore for controlfile from 20/12/15 works.

I would be grateful for any hints or ideas what can I do at this point.

Labels (1)
Tags (3)
0 Kudos
1 Solution

Accepted Solutions
ble1
5 Iridium

Re: Recovering rman data from Networker

Jump to solution

Catalog can be populated via rman - your DBA should know that.  Data stream by RMAN is encapsulated and therefore only RMAN can read it (well, it can understand how to handle the stream).

If you want to restore something which is in catalog (or not), but expired on NW side, you must extend b/r policy, restore index and you are fine from there - that is correct.

I think it is essential to to have both catalog and NW retention synced.  In the past we used to use cleanup of catalog for stuff older than retention on backup server side, but we stopped with that since growth and impact are no so visible and it is good archive to keep audit data as well.

0 Kudos
18 Replies
crazyrov
3 Zinc

Re: Recovering rman data from Networker

Jump to solution

Have you restored the index files for the backups that you are trying to recover ?

Since this is backed up using NMO/NMDA you will not be able to restore the savesets usinng saveset recovery.

I suggest you restore the data using index, ensure that the file that backup file that you are trying to access is present in the backups. You could confirm this using nsrorainfo - if you are using NMDA or incase you had backed this up using NMO use nsrinfo to get the information on the files that were backed up under this saveset.

ble1
5 Iridium

Re: Recovering rman data from Networker

Jump to solution

From what I see you have E state so I assume all backup pieces are expired.  You need to bring them back to be available for restore meaning extend browse and retention and then restore index before you can move on to RMAN restore.

0 Kudos
tolejniczak
1 Nickel

Re: Recovering rman data from Networker

Jump to solution

Thanks for your responses.

As far as I know I have to mark saveset as notrecyclable before I can extend browse/retention times. When I tried to do that I got an error:

# /usr/sbin/nsr/nsrmm -S 1237518202/1704121005 -s bdlbackup -o notrecyclable

Mark save set copy 1237518202/1704121005 as not recyclable? y

Permission denied, user 'root' on 'sd1-bak' does not have 'Operate NetWorker' privilege to perform this operation.

The same error occurs for nsrinfo.

When trying to give administrator privileges to root@sd1-bak the following error occurs:

root@bdlbackup # nsraddadmin -u root@sd1-bak

39078:nsraddadmin: RAP error: Host name your-f261187dc7 is not valid

What does it mean?

0 Kudos
ble1
5 Iridium

Re: Recovering rman data from Networker

Jump to solution

What is your-f261187dc7?  Do you use DHCP?  If not, check DNS/hosts to make sure name resolution is in place.

0 Kudos
tolejniczak
1 Nickel

Re: Recovering rman data from Networker

Jump to solution

In meanwhile I wanted to try recovery procedure on more current saveset.

nsrinfo -vV -s bdlbkp sdbkp

scanning client `sdbkp' for all savetimes from the backup namespace on server bdlbkp

0 objects found

nsrinfo -vV -n oracle -s bdlbkp sdbkp

scanning client `sdbkp' for all savetimes from the backup namespace on server bdlbkp

UNIX file `c-2891241567-20160126-03' size=55838552, off=0, app=oracle(25), date=1453810756 Tue Jan 26 13:19:16 2016, (unknown fid), file size = 0

...

scanning client `sdbkp' for all savetimes from the backup namespace on server bdlbkp

UNIX file `db_SDDB_213567_802561214' size=262328, off=0, app=oracle(25), date=1453426280 Fri Jan 22 05:18:00 2016, (unknown fid), file size = 0

...

Is seems RMAN backups are in index of oracle namespace.

Now when I try to do:

/usr/sbin/nsr/recover -s bdlbkp -x oracle

I got:

39584:recover: No files have been backed up for this client. No index was found on the server for the path / on this client.

I tried also:

/usr/sbin/nsr/recover -a -s bdlbkp -x oracle -d /tmp db_SDDB_213567_802561214

Nothing to recover

When I do:

/usr/sbin/nsr/recover -s bdlbkp

It works:

Current working directory is /

recover> ls

(it prints contents of / directory)

How to extract this RMAN backup from tape?

0 Kudos
ble1
5 Iridium

Re: Recovering rman data from Networker

Jump to solution

RMAN backup can only be restored by RMAN restore.  If you never did it, you are not DBA.  In that case I suggest you talk to DBAs - they can easily make RMAN restore script and run it.

tolejniczak
1 Nickel

Re: Recovering rman data from Networker

Jump to solution

That's the problem - RMAN is no longer aware about those backups existence.

An Oracle guy in my company suggested restoring them from tape to disk and then registering their copies on disk in RMAN catalog. That would allow RMAN restore from data on disk.

As far as I understand now:

* There is no way to do that (extract RMAN data from tape to disk without using RMAN, i.e. with networker commands alone).

* I have to extend browse/retention date of those backups, change their state to notrecyclable, and then restore online file index for those backups (by nsrck -L7 -t ...) . Then I can try to register them in RMAN catalog and perform RMAN recovery from tape. If that doesn't work there is nothing more I can do.

Is that correct?

0 Kudos
edino1
1 Nickel

Re: Recovering rman data from Networker

Jump to solution

Hi,

it is not possible to extend browse or retention policy of expired SSID. You have to load tape with needed SSID to drive and use command scanner for rebuild of media DB entries and indexes . See more details in man page of scanner command...

You will need to restore control file, which contains data about your needed backups. I hope that you have enabled autobackup of control file in your backup RMAN script.

0 Kudos
ble1
5 Iridium

Re: Recovering rman data from Networker

Jump to solution

Catalog can be populated via rman - your DBA should know that.  Data stream by RMAN is encapsulated and therefore only RMAN can read it (well, it can understand how to handle the stream).

If you want to restore something which is in catalog (or not), but expired on NW side, you must extend b/r policy, restore index and you are fine from there - that is correct.

I think it is essential to to have both catalog and NW retention synced.  In the past we used to use cleanup of catalog for stuff older than retention on backup server side, but we stopped with that since growth and impact are no so visible and it is good archive to keep audit data as well.

0 Kudos