Avamar backups or restores to or from the Data Domain fails or completes with exceptions with error messages mentioning "5341" and "SSL library error"

By default, we use high encryption and Auth module for Data Domain connectivity for version 5.5 (5.4 would still work in this scenario without any issues) and if any customer is using a WAAS (WAN optimization devices such as CISCO WAAS) device in their environment it would cause the backups to fail for these clients, or other security devices that analyze traffic.
 
When DDBoost TCP traffic is passed to a WAAS, TCP data is analyzed and then regenerated to achieve several optimizations. None of these optimizations are helpful for DDBoost.
When the  Client Hello  RPC packet sent by DDBoost is intercepted by the appliance, it never reaches the DDR. This causes the SSL handshake between DDBoost and DDR to fail.
Consequently, DDBoost fails to establish TLS connection to DDR.
 
Since, these WAAS devices/Firewalls conducts optimization based on the TCP port, resolution would be to exclude all DD/Avamar communication ports on these devices.

After upgrading to Avamar 7.1 backups are failing with ddr authentication issues. 

All VM backup policy groups were set to no encryption, since 7.1 auto blocks 27000 they have now changed that to High, also in dataset Data Domain encryption is set to default.

For restores, the operation may restore the file(s) to the specified location on the client.  However, the restored file is corrupted and unusable.

The following error messages can be found the client logs (avtar logs ) on the Avamar Administrator GUI or on the client itself

Sample Output 1 (restore):

         2015-11-12 15:56:48 avtar Error : Data Domain server "avamar.emc.corp" open failed DDR result code: 5341, desc: SSL library error
         2015-11-12 15:56:48 avtar Error : Problem logging into the DDR server index:2
         2015-11-12 15:56:54 avtar Error : Unable to open container file for location media: 2 
         2015-11-12 15:56:55 avtar Error : INTERNAL CYCLE ERROR: Unable to restore data for file 'D:\Text.txt' (code:3 - pax container object not existant).

Sample Output 2: 
 
Sample Output 3: 
 

None

To allow the failing clients backup you can add these flags to disable the Data Domain authentication that is causing the failures mentioned above. 

Avtar is using Authentication and Encryption set to high for this Data Domain connection, please use the following flags in avtar.cmd

--ddr-auth-enabled=false
--ddr-encrypt-strength=none
--ddr-auth-mode=3  (18.1+)

If customer has large number of clients, add the following flags to the Data set to apply this change on all the clients

After logging into the Avamar Administrator GUI:

1. Select Tools menu
2. Manage Datasets
3. Select the Dataset
4. Click Edit button
5. Select Options button from the popup window
6. Select the correct plugin from the "Select Plug-In Type:" drop-down
7. Click More button

Sample Output: 




For Windows File system restore jobs, change the value of the "Encryption method from Data Domain system" option in the dataset from "High" to "None".  This value disables SSL encryption for the restore operation.