Avamar: Virtual Machine backup is slow and times out due to hardware resource bottleneck
概要: An Avamar guest backup of a Virtual Machine runs slowly, due to a hardware resource bottleneck.
この記事は次に適用されます：
この記事は次には適用されません：
この記事は、特定の製品に関連付けられていません。
すべての製品パージョンがこの記事に記載されているわけではありません。
現象
A virtual machine which has an Avamar client installed in the guest operating system experiences erratic or poor performance.
If the backup reaches the end of the backup window without completing, Avamar reports that the backup fails with "time out - end."
Example:-
Below, we review the logs for a client which experienced erratic behavior since it was virtualized.
Below, we demonstrate a fast backup and a slow backup for the same client.
The log of the fast backup shows a peak performance rate of 2,291,881 files per hour. The file and data change rate in this backup is low.
The log status messages show that over the course of the backup, the file scan rate is constant with no fast or slow periods.
CPU utilization is also reasonably constant remaining between 40-60%.
Below let us demonstrate this graphically using an internal Dell tool which parses Avamar client logs and charts the data.
These graphs show that, under good conditions, the backup runs quickly.
The storage system is feeding avtar with files at a rate that keeps the CPU at a moderately high utilisation level.
Next we review the log for the slower backup. This backup is for the same client data but under different virtual hardware loading conditions. We see the files/hour performance is <400,000 files/hour.
Since the change rate is low, similar to the fast backup, the backup behaviors are comparable.
Below the log, a graphical representation shows that the backup performance begins quickly, then slows down (area highlighted within the red box).
The slow files-per-hour performance correlates precisely with the time period 11am to 7pm.
Note how from 11am-7pm CPU utilization drops from well above 40% to around 5-10%.
After 7pm CPU utilization fluctuates from 15-20% and files per hour performance increases, although not as high as during the initial part of the backup.
If the backup reaches the end of the backup window without completing, Avamar reports that the backup fails with "time out - end."
Example:-
Below, we review the logs for a client which experienced erratic behavior since it was virtualized.
Below, we demonstrate a fast backup and a slow backup for the same client.
The log of the fast backup shows a peak performance rate of 2,291,881 files per hour. The file and data change rate in this backup is low.
2015-09-14 00:49:20 avtar Info <5156>: Backup #12 timestamp
2015-09-14 00:49:20, 13,050,798 files, 1,352,497 folders, 9,671 GB (37,919 files, 437.1 MB, 0.00% new)
2015-09-14 00:49:20 avtar Info <6083>: Backed-up 9,671 GB in 341.66 minutes: 1,698 GB/hour (2,291,881 files/hour)
The log status messages show that over the course of the backup, the file scan rate is constant with no fast or slow periods.
CPU utilization is also reasonably constant remaining between 40-60%.
Below let us demonstrate this graphically using an internal Dell tool which parses Avamar client logs and charts the data.
These graphs show that, under good conditions, the backup runs quickly.The storage system is feeding avtar with files at a rate that keeps the CPU at a moderately high utilisation level.
Next we review the log for the slower backup. This backup is for the same client data but under different virtual hardware loading conditions. We see the files/hour performance is <400,000 files/hour.
Since the change rate is low, similar to the fast backup, the backup behaviors are comparable.
Below the log, a graphical representation shows that the backup performance begins quickly, then slows down (area highlighted within the red box).
2015-09-22 07:41:35 avtar Info <5157>: PARTIAL Backup #35 timestamp
2015-09-22 07:41:35, 9,190,850 files, 1,030,683 folders, 8,283 GB (22,390 files, 397.8 MB, 0.00% new)
2015-09-22 07:41:35 avtar Info <6083>: Backed-up 8,283 GB in 1440.19 minutes: 345 GB/hour (382,902 files/hour)
The slow files-per-hour performance correlates precisely with the time period 11am to 7pm.
Note how from 11am-7pm CPU utilization drops from well above 40% to around 5-10%.
After 7pm CPU utilization fluctuates from 15-20% and files per hour performance increases, although not as high as during the initial part of the backup.
原因
The dataset for our 'example client' is predictable and the change rate is negligible. The client performs a similar amount of work each day,
The erratic performance observed is due to contention for shared hardware resources used by Avamar whilst scanning files.
In the example shown above, hardware resources are either in high demand or are throttled for the client during peak business hours.
Behavior depends on available resources. The most likely resource to be bottlenecked is random seek I/O requests on the storage device where the data is located.
If CPU usage spikes during peak hours, the bottleneck may be a shortage of CPU shares available to the virtual machine.
The erratic performance observed is due to contention for shared hardware resources used by Avamar whilst scanning files.
In the example shown above, hardware resources are either in high demand or are throttled for the client during peak business hours.
Behavior depends on available resources. The most likely resource to be bottlenecked is random seek I/O requests on the storage device where the data is located.
If CPU usage spikes during peak hours, the bottleneck may be a shortage of CPU shares available to the virtual machine.
解決方法
- Run the backup during off-peak hours and (if possible), when the client is not heavily occupied with its regular duties.
- Ensure that the virtual machine hypervisor is not enforcing CPU, memory, or I/O limitations at the time of the backup window
- Avoid overlapping Avamar backups with periods of high activity where contention for shared storage is high.
- For low change rate applications, consider using image-based backup. The image backup proxy appliance backs up only modified blocks. It does not scan every file in the dataset so has lower I/O requirements.
その他の情報
For more information about how to investigate Avamar backup performance issues, see;
For more information about analyzing and interpreting avtar status messages;
For more information about analyzing and interpreting avtar status messages;
対象製品
Avamar製品
Avamar文書のプロパティ
文書番号: 000164580
文書の種類: Solution
最終更新: 10 2月 2025
バージョン: 4
質問に対する他のDellユーザーからの回答を見つける
サポート サービス
お使いのデバイスがサポート サービスの対象かどうかを確認してください。