Avamar 6.1 agent multi-stream

The available performance tuning options depend on the size and change rate of the dataset, the number of independent volumes, the number of files on the filesystem and the properties of the network connection between the client and the server. It's also very important to ensure that the cache files are sized appropriately since most of the performance gain for file system backups (~90%) comes from the file cache.

To the best of my knowledge, multi-streaming on HPUX requires multiple avtar instances. Configuring multiple avtars is the "nuclear" option for performance tuning and it is only effective if there are multiple independent volumes (otherwise you end up with spindle contention and your performance will decrease rather than increase). It would be best to work with support to evaluate what performance tuning options are most appropriate for this particular client. I would recommend opening a service request for this.

In the best case (i.e. there is a low change rate, there are fewer than 10 million files and the client is backing up through the LAN), it's just a matter of getting through the initial backup. Incrementals on low change rate systems are extremely fast since the client only has to "stat" the unchanged files, it never has to re-read them.

We have 15 remote sites and data size varies from 100-400 GB. files are anyways not in millions though.
do i run MULTIPLE avtar.exe or avtar.bin with options to use different cache locations.

initial backups are running since 2 months and all remote sites are critical, thought they are 1/2 per site.

avmaint

If you're seeing backups time-out and unable to complete within the daily window focus on each client in turn and determine where the bottleneck is.  Provided the scope of the data isn't too great, treating the bottlenecks may enable the client to successfully complete a backup within the available window.

If you have remote sites which have successfully completed an initial backup you're probably going to be most concerned with the amount of changed data which needs to get sent over the WAN.  If those clients haven't yet managed to seed their data (initial backup) to Avamar, the right people to involve will be your local EMC team (account manager & Professional Services)

As Ian mentioned, configuring multiple parallel avtars is an extreme workaround.  It's rarely necessary or appropriate and actually requires product management approval via your account manager. 

Focus on finding where the bottlenecks lie.  The following guide should help.  It also points to various other related articles

• KB 163552 - How to analyse Avamar client backup performance and identify performance bottlenecks

Nick, thanks for the KB article which indeed is elaborate and exhaustive.

Backing up data during Avamar maintenance is neither good for backups not maintenance. Do you see a need for an option for administrator to pause backups in such remote backup cases?

I did suggest a single node Avamar server reachable by other locations.

Also, is it possible to add a node which is remote from Avamar server but local to backup clients?

and backup clients backup to this Avamar node ONLY.

Ex: Util node is in US and storage nodes are one in EU, other in AUS and another in Africa?

Backing up data during Avamar maintenance is neither good for backups not maintenance. Do you see a need for an option for administrator to pause backups in such remote backup cases?

Occasional or limited backups during the maintenance window is generally not recommended.  Sometimes customers choose to use a separate Avamar server for client backups which need to occur during the maintenance window, for example Desktop/Laptop clients which are often switched off during the evening/night when server class machines tend to run backups.

is it possible to add a node which is remote from Avamar server but local to backup clients?

and backup clients backup to this Avamar node ONLY.

It's common for customers to have a smaller, local Avamar server, perhaps even an Avamar virtual edition which backs up clients based in remote offices.  That Avamar server (which is local to the remote clients) may replicate back to a central Avamar server, if desired.

attached is backup log from client, any clues from that, backup is running since 2nd Aug. I did go through the log and compare with the scenarios mentioned in KB, like CPU etc , but didnt see any significant

can someone quickly help us ?

I understand from the statistics section that JPG and media content is more in backups.

The attached log shows a 100% file cache miss rate which seems to indicate that this client has never successfully backed up. In addition, avtar is at 0% CPU utilization for nearly the entire 4 day run of the backup. It looks to me like this client is CPU starved but you should use performance monitoring tools like perfmon or Process Monitor to confirm.

Generally if the CPU utilization is consistently low and the performance is bad, it's likely that some other process is using a lot of CPU time while the backup is running. The avtar process runs at below normal priority so it's easy for it to end up CPU starved if another higher priority process is using a lot of CPU.

This isn't the only possible cause of low avtar CPU utilization which is why I recommended using OS tools to find the bottleneck.