Avamar Client：由于停顿问题，IDPA Linux 客户端映像备份完成但出现异常

由于停顿问题，IDPA 系统的 Linux 映像备份处于“Completing but Exception”状态。

此示例虚拟机是 IDPA 解决方案附带的内部 ACM 虚拟机。

从备份的 VMware linux 映像日志中，可以看到以下内容：

avvcbimage Info <40726>: VM Config Info: VMX version 4(F), ctk enabled(T), fault tolerant (F), Guest OS SUSE Linux Enterprise 12 (64-bit), Cloud UUID = NoUUID. version=9
avvcbimage Info <40727>: VM Guest Info: GuestOS Family = linuxGuest, Guest Full Name = SUSE Linux Enterprise 12 (64-bit), Guest OS State = running, Tools Running = guestToolsRunning,  Tools Version Status = guestToolsUnmanaged
avvcbimage Info <42650>: Creating snapshot 'Avamar-<epoch_time><client_id>', quiesceFS=1
avvcbimage Info <41136>: SnapshotManager: createSnapshot for vmMoref = vm-21
avvcbimage Info <42182>: SnapshotManager: POST to /snapshotmanager/vm/vm-21/action/createSnapshot, payload({
  "snapshotName": "Avamar-<epoch_time><client_id>",
  "snapshotDesc": "Created by Avamar workorder Management_VM_Backup-Management_VM_Backup-<epoch >  on <date>",
  "quiesceFileSystem": "true"
...
avvcbimage Warning <19733>: vSphere Task failed (quiesce, snapshot error=45): 'An error occurred while saving the snapshot: Failed to quiesce the virtual machine.'.
avvcbimage Error <17775>: Snapshot 'Avamar-<epoch_time><client_id> ' creation for VM '[vsanDatastore] DataProtection-ACM/DataProtection-ACM.vmx' task creation encountered a quiesce problem
avvcbimage Warning <40710>: The VM could not be quiesced prior to snapshot creation and this backup will not be used as a base for subsequent CBT backups if successful.
avvcbimage Warning <40711>:     Consider using the pre-freeze and post-thaw scripts to facilitate VM quiescing
avvcbimage Warning <41146>:     This error might caused by the pre-freeze script returning a non-zero error code.
...

第一个快照任务失败后，备份将继续创建第二个快照，但禁用停顿功能。

avvcbimage Info <14631>: VM Disk Consolidation for '[vsanDatastore] DataProtection-ACM/DataProtection-ACM.vmx' task still in progress, sleep for 3 sec
avvcbimage Info <42650>: Creating snapshot 'Avamar-<epoch_time><client_id>', quiesceFS=0
avvcbimage Info <41136>: SnapshotManager: createSnapshot for vmMoref = vm-21
avvcbimage Info <42182>: SnapshotManager: POST to /snapshotmanager/vm/vm-21/action/createSnapshot, payload({
  "snapshotName": "Avamar-<epoch_time><client_id>",
  "snapshotDesc": "Created by Avamar workorder Management_VM_Backup-Management_VM_Backup-<epoch> on <date>",
  "quiesceFileSystem": "false"

备份完成，但显示为已完成，但有异常（代码 10020）

在工单日志 （.alg） 中，可以看到以下内容：

2019-04-03 00:12:42 Management_VM_Backup-Management_VM_Backup-<epoch>Warning <0000>: Workorder "Management_VM_Backup-Management_VM_Backup-epoch" non-zero exit status 'code 170: completed with errors, client log should be examined'

1. 要查看导致停顿失败的原因，请为 vmtools 设置调试日志记录。
   按照 https://kb.vmware.com/s/article/1007873 中的步骤在来宾操作系统上设置调试日志记录，然后重新启动 VMware 工具。

2. 执行新备份以重新创建停顿故障。

3. 查看调试工具日志（停顿信息应位于 vmsvc 日志中并具有 [vmbackup] 前缀）。

在此示例中，问题是由部分删除的 /var/log/brs 目录引起的。由于后台进程仍然打开了该目录，因此无法完全删除此目录。此部分删除的文件夹可防止 Linux 文件系统停止使用 ioctls 函数进行响应。

[   debug] [vmbackup] Submitted backup start task.
[   debug] [vmbackup] *** VmBackupSyncDriverStart
[   debug] [vmsvc] SyncDriver: Skipping remote filesystem, name=systemd-1, mntpt=/proc/sys/fs/binfmt_misc.
[   debug] [vmsvc] SyncDriver: Calling backend 0.
[   debug] [vmsvc] SyncDriver: Freezing using Linux ioctls...
[   debug] [vmsvc] SyncDriver: opening path '/var/lib/named/proc'.
[   debug] [vmsvc] SyncDriver: freezing path '/var/lib/named/proc'.
[   debug] [vmsvc] SyncDriver: freeze on '/var/lib/named/proc' returned: 95 (Operation not supported)
[   debug] [vmsvc] SyncDriver: opening path '/var/log/logbundles'.
[   debug] [vmsvc] SyncDriver: freezing path '/var/log/logbundles'.
[   debug] [vmsvc] SyncDriver: successfully froze '/var/log/logbundles'.
[   debug] [vmsvc] SyncDriver: opening path '/var/log/brs (deleted)'.
[   debug] [vmsvc] SyncDriver: failed to open '/var/log/brs (deleted)': 2 (No such file or directory)
[ warning] [vmbackup] Error freezing filesystems.

更仔细地检查目录 /var/log/brs，可以看到通过其他命令打开此目录时出现问题。像 lsof 和 stat 也显示问题。

IDPA-ACM:/var/log # lsof|grep brs
lsof: WARNING: can't stat() xfs file system /var/log/brs (deleted)
      Output information may be incomplete.

IDPA-ACM:/var/log # ls -al
drwxr-xr-x 0 root     root            6 Feb 27  2018 brs

已重新启动 ACM 虚拟机以终止后台 ACM 进程。重新启动后，ACM 进程重新创建了 /var/log/brs 目录，备份能够完成，而不会出现进一步的停顿问题。

IDPA-ACM:/var/log # ls -al
drwxr-xr-x 2 root     root            6 Apr 15 10:30 brs

确保按照上述知识库文章操作，删除客户端上 vmtools 的调试日志记录，并重新启动 vmtools 服务。