Avamar: NDMP backup canceled by the scheduler when exceeding the backup window

Summary: If Avamar Network Data Management Protocol (NDMP) jobs exceed the schedule limit, the Management Console Service (MCS) cancels them. The log shows avndmp "Cancel request from MCS" and avtar FATAL "WATCHDOG TIMEOUT (cancellation exceeded 600 secs)." Increase the window. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

 Observed Backup Failures.

Symptoms that appear when an Avamar NDMP backup is canceled by the MCS service because it exceeds the scheduled backup window.

  • Backups are listed as Failed in the Avamar Administration GUI → Activity window.
  • NDMP client logs contain multiple sections, including an avndmp section, a progress section, and separate avtar logs for each volume.
  • Individual avtar logs may end with a fatal watchdog timeout, for example:
2016-08-15 02:01:53 avtar FATAL <12477>: WATCHDOG TIMEOUT (cancellation exceeded 600 secs): Exiting with code=163

 

  • The avndmp log shows a graceful termination request from MCS, such as:
2016-08-15 01:51:42 avndmp Info <9772>: [avndmp_ctl_sup] Starting graceful (staged) termination, Process cancel request from MCS (wrap-up stage)

 

  • The avtar logs can display a cancellation message that appears to originate from the NetApp filer, for example:
2016-08-15 01:36:49 avtar Info <7061>: Canceled by '7003-Netapp Filer' - exiting...

 

  • Status lines continue after the cancellation, indicating ongoing processing. For example:
2016-08-15 01:51:49 avtar Info <8688>: Status 2016-08-15 01:51:49, 46,809,759 files, 4,218 GB (46,809,759 files, 0 bytes, 0.00% new) 7852MB 100% CPU (1 open files)

 

  • The combination of "Canceled by…" and "WATCHDOG TIMEOUT" messages can be misleading, suggesting the NetApp filer terminated the job when the trigger is the backup scheduler.

Cause

Backup Scheduler Cancellation Due to Exceeded Backup Window

The backup exceeds the time allocated in its schedule. When this occurs, the Management Console Service (MCS) sends a cancel request to the avndmp process, which then initiates a graceful termination of the backup job.

  • Primary indicator in the avndmp log: 
    avndmp Info <9772>: [avndmp_ctl_sup] Starting graceful (staged) termination, Process cancel request from MCS (wrap-up stage)
  • Resulting message in the avtar log: 
    avtar FATAL <12477>: WATCHDOG TIMEOUT (cancellation exceeded 600 secs): Exiting with code=163
  • Misleading secondary message suggesting a NetApp filer cancellation: 
    avtar Info <7061>: Canceled by '7003-Netapp Filer' - exiting...
     

    This line appears in the log but does not reflect the true source of the cancellation; the actual trigger is the MCS‑initiated timeout.

Therefore, the root cause is an insufficient backup window that forces the scheduler to terminate the job once the allotted time is exceeded.

Resolution

Adjust Backup Schedule to Prevent MCS Cancellation.

Step‑by‑step procedure

CAUTION: Reducing the backup window may cause future cancellations. Always ensure that the window exceeds the longest observed backup duration.
CAUTION: Changing the schedule affects all jobs that share the same client schedule. Review other client schedules to avoid unintended conflicts. /div>
NOTE: If you are unsure about the appropriate window size or encounter repeated cancellations after adjustment, contact Dell Support before making further changes.
  • Open the Avamar Administration UI and go to Activity to identify the failed NDMP backup.
  • Locate the most recent successful backup for the same client. Note its start time, end time, and total duration.
  • Compare the duration of the successful backup with the failed run. If the failed backup exceeds the configured window, the MCS scheduler issues a cancel request.
  • Increase the backup window for the affected client:
    • In the Avamar Administration UI, select the client.
    • Choose Schedule → Edit.
    • Extend the Start Time or End Time so the window is longer than the longest observed duration plus a safety margin (For example, 15 %).
    • Save the changes.
  • If the backup is a level‑0 (full) NDMP job, allow overtime until the next scheduled job completes.
  • If the backup is an incremental or synthetic‑full job, verify the avtar status lines for outliers
  • 2016-08-15 01:51:49 avtar Info <8688>: Status 2016-08-15 01:51:49, 46,809,759 files, 4,218 GB (46,809,759 files, 0 bytes, 0.00% new) 7852MB 100% CPU  (1 open files)
  • Compare these values with previous successful runs.
  • Review the avndmp log for the cancel request line: 
    2016-08-15 01:51:42 avndmp Info <9772>: [avndmp_ctl_sup] Starting graceful (staged) termination, Process cancel request from MCS (wrap‑up stage)

    The timestamp confirms the scheduler‑initiated cancellation.
  • After adjusting the schedule, run a manual backup to confirm the job completes without cancellation.
  • Verification
  • Confirm that the backup appears as Successful in the Avamar Administration UI Activity window.
  • Check the avndmp log for the absence of a "Process cancel request from MCS" entry.
  • Review the final avtar log lines for a normal termination message, such as: 
    2016-08-15 02:01:53 avtar Info <0000>: Entering the 'final' phase of termination, timer trigger

Affected Products

Avamar

Products

Avamar, Avamar Plug-in for NDMP
Article Properties
Article Number: 000064105
Article Type: Solution
Last Modified: 23 Oct 2025
Version:  4
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.