Avamar: Exchange VSS plugin - How to collect logs and environment information for troubleshooting

The purpose of this KB is to give instructions to collect logs and environment information for troubleshooting Avamar Exchange VSS plugin issues.

This KB article should be used as first step by any engineer that needs to troubleshoot Avamar Exchange VSS plugin issues.
In some occasions the issue would be resolved by simply trying to answer the below questions.

Collecting Exchange VSS Plugin logs:

1. Before collecting the logs, it is advisable to first enable log debugging by following the Avamar: Enabling debug logging for Exchange VSS plugin/

After enabling the log debugging, reproduce the issue or wait for the next issue occurrence. 

2. At this point collect the entire avamar folder "C:\Program Files\avs\var", where "C:\Program Files\avs" is the Avamar installation folder.

Once you have copied the required files you can compress them all into one .zip file.

Even though it is recommended to collect all the logs as described above we might only use some of them for the troubleshooting depending on the type of issue we are focusing on.

Log Example:

3. Run the Avamar Config Checker tool and collect the generated file.

For full instructions on the Avamar Config Checker tool refer to the Technical Note EMC Avamar Config Checker for Microsoft Windows

4. Collect the Windows Event logs (both System and Application logs).

• Application Event logs contain Exchange Server events for I/O activity and backup & restores activities.
• System Event logs contain Windows OS events along with the VSS writer's activities that may help to better understand the healthy state of the Windows server during the Avamar plugins operations.

If you need assistance on understanding or collecting the Windows Event logs the Microsoft TechNet forum provides detailed articles, we encourage you to check them out.

Collecting Environment Information:

• Windows server version (with service pack):
• Exchange server version (with service pack):
• Avamar VSS plugin version:
• Is the configuration in standalone or in cluster setup?
• If cluster how many nodes as part of it?
• Avamar user account name use to start avagent service? 
• Does the Avamar user account have required roles/privileges?

• For Exchange 2010/2013 it requires the following permissions:
  • Avamar Roles
  • Exchange Trusted Subsystem
  • Domain Users
  • Exchange Servers

• For Exchange 2007 it requires the following permissions:
  • Domain Users
  • Domain Admins
  • Backup Operators
  • Exchange Servers
  • Exchange Organization Management
    
    

Collecting Extra Information:

• Which of the following types of issues matches your troubleshooting?
  • Installation
  • Configuration
  • Backup
  • Restore
  • Performance
• Have you found any KB article that may describe the issue?
• Was this working before (Y/N)? 
• If yes, what changed: 
• Is the setup on physical machine or VM?
• Can you browse/backup file system volumes using FS plugin via Avamar GUI (Y/N)?
• Can you browse/backup Exchange environment using Exchange VSS plugin via Avamar GUI (Y/N)?
• Did you use native tools like Windows Server Backup (WSB) utility to backup the database/s and verify if the issue is outside Avamar scope?
  
   

Important Notes:

• The reason why we recommend to collect all logs from the /var folder regardless, is because we can have a bigger picture of the issue and it would be handy in case we need to escalate the issue, this is a better way of troubleshooting.
• If you are troubleshooting a cluster environment you will require to collect the local backup logs from each exchange node /var as well as the virtual client logs from the cluster shared folder.
• Make sure you will exclude from copied "\var" folder any sub folder and all the files with extention .dat which are cache files that are not required for troubleshooting and if collected they will cause delays in the transfers and upload processes due to their considered size.

Related KB Articles that may be useful:

• Avamar: How to gather logs to troubleshoot backup and restore issues (000081546)
• Avamar: How to configure client log retention and how to limit the amount of space Avtar logs consume on a client (000060840)
• Avamar: Enabling debug logging for Exchange VSS plugin (000021941)