Avamar: VSS Disabling User Profile Gathering for Windows Clients

Windows Security logs indicate that avtar.exe is accessing every user profile on a client.

• For active user profiles, the entries look like:

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          5/27/2017 4:00:07 PM
Event ID:      4648
Task Category: Logon
Level:         Information
Keywords:      Audit Success
User:          N/A
Computer:      CNCSD1C.corp.emc.com
Description:
A logon was attempted using explicit credentials.

Subject:
Security ID: SYSTEM
Account Name: CNCSD1C$
Account Domain: CORP
Logon ID: 0x3e7
Logon GUID: {00000000-0000-0000-0000-000000000000}

Account Whose Credentials Were Used:
Account Name: testuser
Account Domain: CORP
Logon GUID: {1d662ff0-b57a-9c60-620c-b7f5c70ad1df}

Target Server:
Target Server Name: localhost
Additional Information: localhost

Process Information:
Process ID: 0x1544
Process Name: C:\Program Files\avs\bin\avtar.exe 

-----

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          5/27/2017 4:00:07 PM
Event ID:      4624
Task Category: Logon
Level:         Information
Keywords:      Audit Success
User:          N/A
Computer:      CNCSD1C.corp.emc.com
Description:
An account was successfully logged on.
 
Subject:
Security ID: SYSTEM
Account Name: CNCSD1C$
Account Domain: CORP
Logon ID: 0x3e7
 
Logon Type: 3
 
New Logon:
Security ID: CORP\testuser
Account Name: testuser
Account Domain: CORP
Logon ID: 0x8150fc1
Logon GUID: {cac983ee-8bf7-3789-896f-c9be1e852ead}
 
Process Information:
Process ID: 0x1334
Process Name: C:\Program Files\avs\bin\avtar.exe

• For expired user profiles, it looks like:
   

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          5/27/2017 12:51:58 PM
Event ID:      4625
Task Category: Logon
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      CNCSD1C.corp.emc.com
Description:
An account failed to log on.

Subject:
Security ID: SYSTEM
Account Name: W8001DB03$
Account Domain: INTERNAL
Logon ID: 0x3e7
 
Logon Type: 3
 
Account For Which Logon Failed:
Security ID: NULL SID
Account Name:                     
Account Domain:                 
 
Failure Information:
Failure Reason: The specified user account has expired.
Status: 0xc0000193
Sub Status: 0xc0000193
 
Process Information:
Caller Process ID:  0xe7c
Caller Process Name: C:\Program Files\avs\bin\avtar.exe

• For disabled user profiles, it looks like:
   

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          5/27/2017 12:51:58 PM
Event ID:      4625
Task Category: Logon
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      CNCSD1C.corp.emc.com
Description:
An account failed to log on. 

Subject:
Security ID: SYSTEM
Account Name: W8001DB03$
Account Domain:  INTERNAL
 Logon ID:  0x3e7
 
Logon Type: 3
 
Account For Which Logon Failed:
Security ID:  NULL SID
Account Name:                     
Account Domain:                 
 
Failure Information:
Failure Reason:  Account currently disabled.
Status: 0xc000006e
Sub Status: 0xc0000072
 
Process Information:
Caller Process ID:  0xe7c
Caller Process Name: C:\Program Files\avs\bin\avtar.exe

• Entries such as the following can also be seen:
   

Log Name:      Security
Source:        Microsoft-Windows-Security-Auditing
Date:          5/27/2017 12:51:58 PM
Event ID:      4625
Task Category: Logon
Level:         Information
Keywords:      Audit Failure
User:          N/A
Computer:      CNCSD1C.corp.emc.com
Description:
An account failed to log on. 

Subject:
Security ID: 
Account Name: testuser
Account Domain: CORP
Logon ID: 0x3e7

Logon Type: 3

Account For Which Logon Failed:
Security ID: NULL SID
Account Name:        
Account Domain:        

Failure Information:
Failure Reason:  Error occured during Logon.
Status: 0xc000018b
Sub Status: 0x0

Process Information:
Caller Process ID: 0x1544
Caller Process Name: C:\Program Files\avs\bin\avtar.exe

• The following is a list of common statuses that may be encountered:
   

   
  Status Code	Description
  0XC000005E	There are currently no logon servers available to service the logon request.
  0xC0000064	User logon with misspelled or bad user account
  0xC000006A	User logon with misspelled or bad password
  0XC000006D	This is either due to a bad username or authentication information
  0XC000006E	Unknown user name or bad password.
  0xC000006F	User logon outside authorized hours
  0xC0000070	User logon from unauthorized workstation
  0xC0000071	User logon with expired password
  0xC0000072	User logon to account disabled by administrator
  0XC00000DC	Indicates the Sam Server was in the wrong state to perform the desired operation.
  0XC0000133	Clocks between DC and other computer too far out of sync
  0XC000015B	The user has not been granted the requested logon type (aka logon right) at this machine
  0XC000018C	The logon request failed because the trust relationship between the primary domain and the trusted domain failed.
  0XC0000192	An attempt was made to logon, but the Netlogon service was not started.
  0xC0000193	User logon with expired account
  0XC0000224	User is required to change password at next logon
  0XC0000225	Evidently a bug in Windows and not a risk
  0xC0000234	User logon with account locked
  0XC00002EE	Failure Reason: An Error occurred during Logon
  0XC0000413	Logon Failure: The machine you are logging onto is protected by an authentication firewall. The specified account is not allowed to authenticate to the machine.
  
  

For a full list, see Error ode ntstatus.h (External Link) 
These entries are found in the Security Log for every user profile on the client machine every time the backup runs.

At the end of each backup, the Plugin spawned avtar process gathers information for every user profile on the client.

• In the avtar log, the following line can be found (notice, the number varies depending on the number of profiles):

avtar Info <11035>: Reading 14 user profiles
avtar Info <11036>: Done reading user profiles

• This gathering of profiles happens at the end of every avtar session on a Windows machine. It happens not only at the end of a Windows File System backup (avtar), but also every time a different plug-in like avexvss (Exchange), avsql (SQL), avvss (VSS) spawns an avtar.exe process.
•  If a Windows VSS backup spawns three avtar processes to backup various volumes, the profiles are gathered three times and adds to the overhead times.
• Although user profile gathering is supposed to be quick process, in some rare instance like orphaned security identifier (SID) entries it takes a long time impacting Avamar performance. Example of such logged entry:

2017-05-25 04:34:18 avtar Info : Reading 37 user profiles

Followed over two hours later by:

2017-05-25 06:50:34 avtar Info : Done reading user profiles

• Profile gathering at the end of the backup can even fail when invoking "AuthzInitializeContextFromSid":

2023-10-13 09:51:21 avtar Warning <16147>: AuthzInitializeContextFromSid failed: 2

More details about the use of this API in profile gathering is located at:

https://learn.microsoft.com/en-us/troubleshoot/sql/reporting-services/call-authzinitializecontextfromsid-api-fails

In such cases, some SIDs had the corresponding username entries missing and avtar stuck or failed processing these orphaned SIDs. This can happen when deleting user accounts but not deleting the corresponding user home directory.

This profile gathering is turned on by default, but is only used for desktop or laptop (DTLT) restores. For each user profile, avtar obtains all groups the user belongs to in order to determine whether the user is a local administrator. This information is used to determine which files the logged in user can see and restore using the DTLT web interface.

Although these security entries can be safely ignored, profile gathering can be disabled on Windows Server clients. It should not be disabled on desktops or laptops if the DTLT web interface is being used. To disable User Profile gathering, add the following avtar flag in the avtar.cmd file on the client or the associated Dataset.

--x05=65536 

The disabling of profile gathering can be handled in two ways.

1. For a single client:
   1. Create a text file in C:\Program Files\avs\var called avtar.cmd
   2. In the avtar.cmd file, add the following flag:

   3. --x05=65536

   4. This affects all backups on the client, since avtar uses it every time it is started.
2. For multiple clients using a Dataset:
   1. In the dataset, go to the 'Options' tab
   2. Select the appropriate Plug-In type from the dropdown list
   3. Click the 'More' button.
      1. For Windows File System backups:
         1.  Under 'Enter Attribute: Enter x05
         2. Under 'Enter Attribute Value', enter 65536
         3. Then click the + button
      2.  For all other Windows plug-ins:
         1. Under 'Enter Attribute:' Enter [avtar]x05
         2. Under 'Enter Attribute Value', enter 65536
         3. Then click the + button
   4. This must be done for each plug-in type that is part of the dataset and for every dataset that is assigned to a group that the client is a member of.