Avamar: File Level Restore: "Invalid response from the client agent"

Avamar: File Level Restore Fails with "Invalid response from the client agent java.io.EOFException"

Error in the Avamar User Interface (AUI) right after selecting FLR: Invalid response from the client agent java.io.EOFException" 

Error in the Avamar mcserver log:

[RMI TCP Connection(129845)-AVAMAR-SERVER-IP#3792934] com.avamar.mc.util.MCException.logException
WARNING: java.io.EOFException
  at com.avamar.mc.cr.bcConnection.readPreamble(bcConnection.java:1806)
  at com.avamar.mc.cr.Browser.awaitReply(Browser.java:789)
  at com.avamar.mc.cr.BrowserVmBackup.browse(BrowserVmBackup.java:84)
  at com.avamar.mc.burm.BackupRestoreManagerService.getArchiveContentsDocument(BackupRestoreManagerService.java:1145)
  at com.avamar.mc.burm.BackupRestoreManagerService.getArchiveContents(BackupRestoreManagerService.java:886)
  at com.avamar.mc.burm.BackupRestoreManagerService.getArchiveContents(BackupRestoreManagerService.java:722)
  at sun.reflect.GeneratedMethodAccessor12804.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
  at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:61)
  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
  at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
  at com.sun.proxy.$Proxy38.getArchiveContents(Unknown Source)
  at com.avamar.mc.burm.BackupRestoreManagerServiceContext.getArchiveContents(BackupRestoreManagerServiceContext.java:150)
  at sun.reflect.GeneratedMethodAccessor12803.invoke(Unknown Source)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
  at sun.rmi.transport.Transport$1.run(Transport.java:200)
  at sun.rmi.transport.Transport$1.run(Transport.java:197)
  at java.security.AccessController.doPrivileged(Native Method)
  at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
  at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
  at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
  at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
  at java.security.AccessController.doPrivileged(Native Method)
  at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
  at java.lang.Thread.run(Thread.java:750)
[RMI TCP Connection(129845)-Avamar-Server-IP#3792934] com.avamar.mc.cr.BrowserVmBackup.browse
FINE: Client to MCS(Proxy-IPAddr:28009) responded to <browse>.<browse errdesc="Browser ERROR: Invalid response from the client agent java.io.EOFException"></browse>

The Avamar Management Console Server (MCS) randomly tries contacting any registered proxy, which in this case is a dead proxy for the File Level Restore operation.

The MCS log shows the following:

[RMI TCP Connection(129845)-Avamar-Server-IP#3792934] com.avamar.mc.cr.BrowserVmBackup.browse
FINE: Client to MCS(Proxy-IP Address:28009) responded to <browse>.<browse errdesc="Browser ERROR: Invalid response from the client agent java.io.EOFException"></browse>

The above portion of mcserver.log contains "Client to MCS(Proxy-IP Address: 28009." This IP address and port refer to the DEAD proxy.

Instead of sending the Browse request to a live proxy, MCS is sending to a DEAD proxy.

Fix ALL dead proxies or remove them from Avamar.