Question about Windows processes involved in Avamar backup

Working with a customer on backups for a large Windows file server, and am trying to get an idea of performance, so I have the Windows Resource Monitor open and I'm looking at disk operations.

While I see the expected entries for avtar under both "Processes with Disk Activity" and "Disk Activity", I am also seeing what appear to be "associated" operations of some kind for "System" with a PID of "4" - and under the "Disk Activity" section, the System entries seem to be working on the same "\Device\Harddisk\VolumeShadowCopy...\" entries that the avtar processes are also working on.

I understand that the Windows backup uses VSS these days and that Avamar would likely "hit" that shadow copy - I also understand that Windows will likely be working on that shadow copy as well to a degree. What I curious about is that at times, the System processes seem to be a lot more busy with the shadow copy than avtar is - and while this system is being used (as a file server), I wouldn't expect it to be "hit" as much by users as I would expect it to be "hit" by Avamar during a backup.

(and I also understand that System PID 4 is somewhat of a "catch all" process that takes care of a lot of things - but I still can't figure out why at some points it would be showing 4x or more the number of I/Os that avtar would)

Can anyone provide some additional perspective on why there are so many System processes "hitting" the shadow copy as well as the avtar ones?

All comments/feedback appreciated - thanks.

Responses(2)

AE

Avamar Exorcist

462 Posts

1

March 14th, 2018 03:00

Whilst avtar might be a particularly 'demanding' user of the file system as it walks through the files and processes the changed ones, it's just one user. A busy file server may have a large number of other users concurrently writing or updating files. Also be aware that avtar's priority is set to below average so we could reasonably expect non-avtar requests to be served with much more 'gusto'.

I discussed avtar prioritisation with Engineering a while back and documented the following in KB 335284

CPU Process Prioritization
Linux Behavior:

The avagent process automatically sets all processes it spawns to be at a nice level.
In Linux, priority levels range from -20 to 20, where -20 is the highest priority .
By default, processes are spawned with priority 0.
Avtar is created with priority 10, which is much lower than the default priority.

Windows Behavior:

Windows manages priority levels ranging from 1 (lowest) to 31 (highest).
Avtar is created with BELOW_NORMAL_PRIORITY_CLASS, which means that threads owned by avtar by default are created with priority 6.
For reference, threads created by processes on Windows have a default priority of 8.

This is more likely to be a challenge if the data-set is so large or high-change that it can't be backed up within the quiet hours of the day.

Edit: The KB article also refers to Avamar bug 240762 where we discussed with Engineering the possibility of providing a flag to tune avtar's priority to be more or less 'polite'. Unfortunately, although it's possible to change the CPU priority for the process, Engineering found that doing this didn't affect the I/O priority, at least on Windows... and since providing such a flag wouldn't have yielded the desired benefit it was decided to not implement it.

Avtar priority levels aside, exhausting all the regular options (KB 335029) to hunt out bottlnecks is advised if you haven't already travelled that route.

C

connolly

2 Intern

•

132 Posts

0

March 14th, 2018 07:00

Thank you for the reply - very good information.

FWIW, in this particular situation, it isn't so much a case of wanting a higher priority for avtar - it is more a case of understanding whether avtar is "performing at its full potential" in terms of what else might be running on the system, especially relative to what else might be accessing the same disks as avtar.

I have been using the KB listed to investigate the performance issues, but unfortunately in many cases, we don't just have to prove that the Dell EMC side isn't the issue - we have to point out what the actual issue is in order for the customer to acknowledge our findings that it wasn't the Dell EMC side.

View All

No Events found!