Cloning Scheduling?

I'll try to make this short. We don't use the Auto Clone feature because NetWorker isn't smart enough to load the source volumes to our storage nodes where the clones take place. We use an EMC CDL to emulate a 30 drive, 512 slot library. This library is controlled by the Networker Server and is responsible for all of our production backups. We have three storage nodes that each control an ADIC Scalar 24 tape library that is connected to our SAN. These three Scalars do all of our clones. Each one of those storage nodes also has two DDS drives from the CDL.
I created a script that:
1) gathers all the SSIDs to be cloned.
2) unmounts any of the source volumes from the tape drives
3) sets the "Location" field of the source volume to the clone storage node.
4) starts the clone
5) sends out a summary report of the clone
We have to use the script so that NetWorker loads the source volumes to the DDS in the clone storage nodes. The clone storage nodes sees the volumes as mounted local to them and sends the data to the Scalars via the SAN. This avoids using our slower production network for the data path.

Thank you all for your help it has been much appreciated. So I guess its cron job time and nsrclone and nsrndmp_clone commands = ).....

Right now we have seperate media pools for backups and what not. Do you recommend just using the Auto-Clone feature from the group side when I run backups or not recommend that option? How is that handled in your environment?

I would suggest scripted cloning. Nevertheless, if you have simple environment automatic cloning might be good start.

Hi Hrvoje,

I didn't know you could change the "read hostname" parameter. What command would I use? I tried depending on the idle device timeout before but it wasn't always reliable. Plus, my script can be used to restart a failed cloned session so the unloading part is useful if you restart right away.

Hi Eric,

With 30 drives and multiple storage nodes I would also use script of course. You can change your script by setting read hostname before nsrclone to force "source node". With DDS unloading devices can be controlled via idle device timeout so you don't need to worry about that.

Let's assume you have 3 nodes with each having ADIC for target and your script is based on functions you then call in main part. Create 3 additional functions like this:

#####   SETTING READ HOSTNAME TO SN1   ###
read_HOSTNAME_SN1()
{
$NSRADMIN -i - << EOF
. type: NSR jukebox;
name: 
update read hostname: 
EOF
}
 
#####   SETTING READ HOSTNAME TO SN2   ###
read_HOSTNAME_SN2()
{
$NSRADMIN -i - << EOF
. type: NSR jukebox;
name: 
update read hostname: 
EOF
}
 
#####   SETTING READ HOSTNAME TO SN3   ###
read_HOSTNAME_SN3()
{
$NSRADMIN -i - << EOF
. type: NSR jukebox;
name: 
update read hostname: 
EOF
}

I assume you have function for each nsrclone set (actually you probably split those to get parallel jobs) per sn so the main part would look like

if [some condition]; then
       [whatever]
       read_HOSTNAME_SN1 > /dev/null 2>&1
       save_NSRCLONE_SN1 > /dev/null 2>&1
       read_HOSTNAME_SN2 > /dev/null 2>&1
       save_NSRCLONE_SN2 > /dev/null 2>&1
       read_HOSTNAME_SN3 > /dev/null 2>&1
       save_NSRCLONE_SN3 > /dev/null 2>&1
       [whatever]
else
       exit $ERROR
fi

Of course, $NSRADMIN points to full path of nsradmin on your system in variable areas somewhere within your script.

One thing to pay attention is when you kill cloning process. "Cloned" instance gets clflags "a" as aborted, but it also increases copies count. I assume you query is against copies=1 if dealing with data on tape so potentially if you don't watch out you may end up with data not cloned if you have stopped cloning while running.

Good stuff. Do you have an example of the cron script that checks to see when the job is done then kick off the backup?

Very nice. I'll try this out in my next script revision that I'm working on. I currently have the clone script scheduled to run in CRON about 10 - 15 minutes after the group completes its backup. When the clone script runs, it checks to make sure that the backup is complete. If not, it will reschedule itself to run again in 10 minutes. In the next revision, I'll update the groups to include a post exec command to start the clone script. That and your "read hostname" code should make it work more efficiently.

Thanks!
Eric

OK_FLAG=`ps -ex | grep -i savegrp | grep -i | grep -v grep | wc -l`
if (( $OK_FLAG < 1 ))
then
< Your cloning commands here >

else #Reschedule clone
print "$GROUP cloning has been delayed 10 minutes" | mailx -s "$GROUP cloning has been delayed 10 minutes" daily_summary_email
at now + 10 minutes <
> /dev/null 2>&1 EOF
fi

Message was edited by:
Eric Locsin

You could simply check for presence of savegrp in case all your backups are scheduled.

Eric,

Question for you where in the script do you specify the 10 minute waiting period for the reschedule of the job?

Ahh totally missed the "at" command.. Sorry about that, thanks again.

This is what you would use:

at now + 10 minutes <
{Your clone script here} > /dev/null 2>&1
EOF