Avamar: SQL multistream backup and restore may fail with "The api was waiting and the timeout interval had elapsed"
Summary: When six or more streams back up or restore ~1 TB databases on SQL Server 2017 2019, the job may stall and time out because VDI uses a semaphore count instead of SetEvent, causing a deadlock in avsql. ...
Symptoms
Large SQL Backup Fails with Multiple Streams
- SQL multistreaming backup and restore may fail with the following error:
2021-09-16 15:01:54 avsql Error <6478>: VDS::Getconfig failed with 'The api was waiting and the timeout interval had elapsed' 2021-09-16 15:01:54 avsql Error <6478>: VDS::Getconfig failed with 'The api was waiting and the timeout interval had elapsed' 2021-09-16 15:01:54 avsql Error <6478>: VDS::Getconfig failed with 'The api was waiting and the timeout interval had elapsed' 2021-09-16 15:01:54 avsql Error <6479>: Timed out. Was Microsoft SQLServer running? - All spawned avtar threads for the
“avsql stack execution”show no progress bytes. - The issue occurs when six or more streams are configured for databases of approximately one terabyte or larger.
- This issue affects Microsoft SQL Server 2017 and 2019 environments. Running the backup with a single stream completes but takes longer.
Cause
Deadlock Caused by VDI Behavior Change
This issue is caused by a change in the Microsoft Virtual Device Interface (VDI) implementation for SQL Server 2017 and 2019. In SQL Server 2016, the avsql plug‑in used the SetEvent parameter to signal completion of the backup configuration. Starting with SQL Server 2017/2019, the VDI layer uses a semaphore count instead. When multistreaming is enabled, the avsql component increments the semaphore count, which leaves the count non‑zero and creates a deadlock that prevents the “avsql” threads from progressing, resulting in the timeout error.
- VDI behavior difference between SQL 2016 (
SetEvent) and SQL 2017/2019 (semaphore). - Avamar
“avsql”component expects theSetEventsemantics. - Multistreaming activates the code path that manipulates the semaphore.
- Deadlock leads to no progress on spawned avtar threads and the timeout message.
Resolution
Permanent Resolution
Upgrade the Avamar backup client and the SQL plug‑in to version 19.10 or later, which includes the code fix. Dell recommends upgrading to the latest supported release (for example, Avamar 19.12.x) to obtain the most recent fixes and enhancements.
Apply the latest cumulative hot‑fix for the installed version:
- For version 19.10.100-166 HF338875
- For version 19.12.100-186 HF953552
Download the required hot‑fix from Dell Support:
See the article Avamar: How to find and download hotfixes, patches, or Installation and Upgrade Package from Dell Support for more information.