Highlighted
CKnapp1
1 Nickel

Overtime and maintenace window

Good morning,

I have a backup that is failing due to hitting the maintenance window. This is after running for 21 hours (almost 2TB of data that hasn't been backed up in over a year). I know I can set the client to allow overtime until successful backup. Will this suspend maintenance window until it completes? Under activity the error code is 10014. Thank you

Also, Avamar version is 6.0.2.153

2014-03-13 06:01:14 avtar Info <5726>: Aborting backup due to error (3:EXC_HFSERROR)

2014-03-13 06:01:14 avtar Error <8610>: Connection canceled from GSAN.

2014-03-13 06:01:14 avtar Info <9901>: Cancel Request being processed (setting code from 536870925 to 536870925)

2014-03-13 06:01:14 avtar Error <8610>: Connection canceled from GSAN.

2014-03-13 06:01:14 avtar Info <9901>: Cancel Request being processed (setting code from 536870925 to 536870925)

2014-03-13 06:01:14 avtar Info <9901>: Cancel Request being processed (setting code from 536870925 to 536870925)

2014-03-13 06:01:14 avtar Info <9901>: Cancel Request being processed (setting code from 536870925 to 536870925)

2014-03-13 06:01:14 avtar FATAL <5399>: Internal error, aborting

Tags (2)
7 Replies
ionthegeek
4 Beryllium

Re: Overtime and maintenace window

Based on the log messages, it looks like this backup is being killed uncleanly when garbage collection starts. If that's the case, the backup is never "sealed". The caches require a recent sealed backup to work to full effect so this client is spending a lot of time redoing the same work every day.

If the backup is being killed when garbage collection starts, it's likely that the system is over the capacity warning threshold. If that is the case, I would not recommend skipping garbage collection. Capacity issues can take a long time to recover from so you don't want to put yourself in that position.

I would actually recommend shortening the schedule for this client to ensure the backup isn't killed by garbage collection every day. If the backup times out cleanly, it seeds the caches for the next day and creates a sealed partial backup. You should see more progress bytes each day until the backup completes. These partials would not be restorable but they would allow the client to take advantage of the caches. Another alternative would be to reduce the dataset down to a portion of the client data you know will succeed and gradually add more data to the dataset each day until the client is backing up all of its data. This alternative would have the same cache advantage and would also produce restorable (though incomplete) backups but would require a slightly higher investment in management time.

0 Kudos
ionthegeek
4 Beryllium

Re: Overtime and maintenace window

Also, I've assumed here that this is a filesystem backup. If this is some kind of database backup (including Microsoft SQL Server, DB2, Oracle, Notes, Exchange, etc.) the advice about partials does not apply since most of the benefit of partials is from the file cache which doesn't contribute much during database backups (the database files have almost always changed so the file cache misses).

0 Kudos
nmc2
2 Iron

Re: Overtime and maintenace window

As per my knowledge client overtime will not suspend maintenance window, but will skip GC.

0 Kudos
CKnapp1
1 Nickel

Re: Overtime and maintenace window

Ian, thank you for the info. I will try changing the schedule to finish an hour before maintenance starts. I understand what you are saying I think. So by letting it partial finish the f/p cache will be able to hopefully catch up and complete the backup in a few days.

0 Kudos
ionthegeek
4 Beryllium

Re: Overtime and maintenace window

by letting it partial finish the f/p cache will be able to hopefully catch up and complete the backup in a few days

Exactly. Happy to help!

Partials are only kept for 7 days in Avamar 6.0 so if the backup doesn't make it to completion by day 7, you'll have to take further steps. If you find that the backup hasn't completed by the end of day 7, I would recommend reducing the size of the dataset and working your way up as I've described above since restorable backups aren't limited to 7 days retention like partials are.

0 Kudos
CKnapp1
1 Nickel

Re: Overtime and maintenace window

Ian,

It looks like setting the backup schedule to stop an hour before maintenance got me a partial. Thanks for the help. Hopefully this can get a usable full backup done in the next 7 days.

2014-03-14 05:00:07 avtar Info <8758>: Cancelled - exiting...

2014-03-14 05:00:07 avtar Info <9901>: Cancel Request being processed (setting code from 0 to 536870931)

2014-03-14 05:00:09 avtar Info <7202>: Backup CANCELLED, wrapping-up session with Server

2014-03-14 05:00:11 avtar Info <11035>: Reading 6 user profiles

2014-03-14 05:00:12 avtar Info <11036>: Done reading user profiles

2014-03-14 05:00:12 avtar Info <9974>:  26 matches for exclude pattern "Temporary Internet Files\".

2014-03-14 05:00:12 avtar Info <9974>:  11 matches for exclude pattern "outlook.ost".

2014-03-14 05:00:12 avtar Info <5157>: PARTIAL Backup #1255 timestamp 2014-03-14 05:00:12, 1,270,396 files, 142,154 folders, 1,377 GB (78,207 files, 106.5 GB, 7.74% new)

2014-03-14 05:00:12 avtar Info <7539>: Label "D 5P-5A-LHM-IAS-1394744400045", scheduled to expire 03/21/14, daily,weekly,monthly backup

2014-03-14 05:00:12 avtar Info <6083>: Backed-up 1,377 GB in 720.20 minutes: 115 GB/hour (105,837 files/hour)

2014-03-14 05:00:12 avtar Info <7883>: Finished at 2014-03-14 05:00:12 Eastern Daylight Time, Elapsed time: 0012h:00m:12s

2014-03-14 05:00:12 avtar Info <6149>: Error summary: 1 error: 9907

2014-03-14 05:00:12 avtar Info <8468>: Sending wrapup message to parent

2014-03-14 05:00:12 avtar Info <5314>: Command failed (1 error, 41 warnings, exit code 10019: Externally cancelled by Administrator)

0 Kudos
ionthegeek
4 Beryllium

Re: Overtime and maintenace window

Good news! Thanks for following up.

0 Kudos