Unsolved
This post is more than 5 years old
5 Posts
0
11532
Avamar hangs on one SQL database
Hello everyone. I am having issues with the backup of a SQL server and I am not able to find the source of the problem. Until yesterday the backup would never complete. It would run for days until I would finally have to cancel the job. After reviewing the Windows application logs I found an error simply stating that this one database backup failed. So I ran another job excluding this specific database and the job completed without incident. So now I have been able to isolate which database is the problem but I cannot find what might be the issue with the database. We are running the following:
Server - Windows Server 2008 R2 Datacenter edition
Avamar - 6.1.1-87
SQL - SQL Server 2008 R2
Windows error log:
Log Name: Application
Source: MSSQLSERVER
Date: 3/6/2013 3:24:21 PM
Event ID: 3041
Task Category: Backup
Level: Error
Keywords: Classic
User: SYSTEM
Computer: XXXXXXXXXXXXXXXXXXX
Description:
BACKUP failed to complete the command BACKUP DATABASE AIMSAudit. Check the backup application log for detailed messages.
avsql log errors:
2013-03-07 11:08:04 avsql Error <6478>: VDS::Getconfig failed with 'The api was waiting and the timeout interval had elapsed' (Log #3)
2013-03-07 11:08:04 avsql Error <6479>: Timed out. Was Microsoft SQLServer running? (Log #3)
2013-03-07 11:08:04 avsql Error <6478>: VDS::Getconfig failed with 'The api was waiting and the timeout interval had elapsed' (Log #3)
2013-03-07 11:08:04 avsql Error <6479>: Timed out. Was Microsoft SQLServer running? (Log #3)
2013-03-07 11:08:10 avtar Info <7202>: Backup CANCELLED, wrapping-up session with Server (Log #1)
2013-03-07 11:08:10 avtar Info <7202>: Backup CANCELLED, wrapping-up session with Server (Log #2)
The database is mounted, SQLServer is running and the database is configured for Full recovery model. Security on this database is the same as all other databases on the same server. When I exclude this database all other databases backup correctly.
Has anyone else experienced this problem? Any ideas on how to continue troubleshooting this problem? Verbose logging has not revealed anything helpful. Thanks.
Steve
sailtodo
5 Posts
0
March 7th, 2013 12:00
That is a good question. I haven't tried that yet but will and post the results ASAP. Thanks for the suggestion.
fdxpilot
143 Posts
0
March 7th, 2013 12:00
Is SQL able to do a native backup of this DB without generating any errors?
sailtodo
5 Posts
0
March 7th, 2013 14:00
The native SQL backup failed so something is going on for sure with this database. Please see attachment for error. Thanks.
1 Attachment
aimsaudit_error.PNG.png
fdxpilot
143 Posts
0
March 7th, 2013 14:00
I'm not a SQL DBA but you can try the suggestion from this MS discussion thread at your own risk: Backup failed: System.Data.SqlClient.SqlError: Backup and file manipulation... must be serialized
and then kill the offending process by SPID.
kranthikumar
5 Posts
2
March 8th, 2013 13:00
Even We have the same issue, the Avamar hungs up, once we kill and re-run on-demand backups it gets successfull and even schedule jobs do well for a while, again later after some days same issue and will repeate killing the jobs....why this happens and is there any final fix for this or something needs to be worked on sql-plugin for improvements..i have seen this issue with may of them as i go through the forums, every says to kill the hung jobs, but no reason why this happens and how to handel this repetition issue.
Opened an SR, waiting for some one to take up the case. will update you guys.
mhelal
11 Posts
0
May 11th, 2013 22:00
also our case here, please update us, and i will open a case for this also, another thing if i cancel the job from the avamar administrator the job will take extra hour and a half to finish its activities on the sql server.
any suggestions !!
ionthegeek
2K Posts
0
May 7th, 2014 09:00
Have you tried running a native SQL backup of this database as fdxpilot mentions above? The avsql plug-in was waiting for the SQL server to perform some task and it appears that this task either didn't complete or didn't complete in a timely manner.
pawankumawat
355 Posts
0
May 7th, 2014 09:00
Hello,
Please give a try to this -
Increase the SQL connection timeout by adding the following line to the file C:\Program Files\avs\var\avsql.cmd:
--sqlstepissuedsecs=60
Regards,
Pawan
davidrvansandt_
3 Posts
0
May 7th, 2014 09:00
And I too having the same issue. I am running Avamar Server 7.0.101-61 and client is v7.0.100-427.
2014-05-07 10:47:23 avsql Error <6478>: VDS::Getconfig failed with 'The api was waiting and the timeout interval had elapsed' (Log #3)
2014-05-07 10:47:23 avsql Error <6478>: VDS::Getconfig failed with 'The api was waiting and the timeout interval had elapsed' (Log #3)
2014-05-07 10:47:23 avsql Error <6479>: Timed out. Was Microsoft SQLServer running? (Log #3)
2014-05-07 10:47:23 avsql Error <6479>: Timed out. Was Microsoft SQLServer running? (Log #3)
2014-05-07 10:47:28 avtar Info <7202>: Backup CANCELED, wrapping-up session with Server (Log #2)
2014-05-07 10:47:28 avsql Error <18491>: SQL plug-in will now terminate due to the following avtar error: 'code 536870925: externally cancelled' (Log #3)
2014-05-07 10:47:36 avsql Error <18491>: SQL plug-in will now terminate due to the following avtar error: 'code 536870925: externally cancelled' (Log #3)
I guess I'll have to resort to opening an SR. Unless somebody has a fix...
davidrvansandt_
3 Posts
2
May 8th, 2014 06:00
So I did some more troubleshooting... this is only a problem on ONE of my servers. I have multiple configured in the same dataset & group that work fine. I tried backing u9p from the Backup, Recovery & Management context and found that the issue has to do with a tempdb database. Even with the exclusion, if tempdb is in the work list, it fails. I've reached out to the DBA to check the server.
bruce.leyton
2 Posts
0
August 24th, 2017 09:00
Hi David, did you tried Pawan recomendations. How much is the filesize of the Database, in my case multiple servers with same policy only one fails with one database, not always, this is because is the largest database in that server (70 GB). Windows Server 2008R2, SQL Server 2008. Avamar 7.2.