Avamar PDM fails right away with "Error while fetching tasks Failed with HTTP error code:404 additional information" HTTP Status 404 - Not Found
Summary: This article explains how to resolve the issue with the Proxy Deployment Manager (PDM) error message: "Error while fetching tasks. Failed with HTTP error code:404 additional information" HTTP Status 404 - Not Found ...
Symptoms
Unable to use Proxy Deployment Manager (PDM) to create recommendations in the Avamar User Interface (AUI) or Avamar Administrator console.
Cause
</pre><p><b>Root Cause</b></p><pre>com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
There is an authentication issue with RabbitMQ.
The 404 error code can be seen in /usr/local/avamar/var/mc/server_log/mcserver.log.0 when using PDM using Avamar Administrator console:
WARNING: Failed (https://localhost:8543/deploymanager/auth/login)(attempt 1 of 3): Failed with HTTP error code : 404
additional information : <!doctype html><html lang="en"><head><title>HTTP Status 404 ▒~@~S Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 ▒~@~S Not Found</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Message</b> The requested resource [/deploymanager/auth/login] is not available</p><p><b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p><hr class="line" /><h3>Apache Tomcat</h3></body></html>
08/25-16:53:06.00956 [MCS Counter monitor thread#168] com.avamar.mc.mon.MCSPerformanceMonitor.monitor
FINE: enter
08/25-16:53:06.00975 [RMI TCP Connection(122)-192.168.33.10#1523] com.avamar.mc.pdm.ProxyDeploymentManagerService.submitJsonWithRetry
WARNING: Failed (https://localhost:8543/deploymanager/auth/login)(attempt 2 of 3): Failed with HTTP error code : 404
additional information : <!doctype html><html lang="en"><head><title>HTTP Status 404 ▒~@~S Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 ▒~@~S Not Found</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Message</b> The requested resource [/deploymanager/auth/login] is not available</p><p><b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p><hr class="line" /><h3>Apache Tomcat</h3></body></html>
08/25-16:53:07.00035 [RMI TCP Connection(122)-192.168.33.10#1523] com.avamar.mc.pdm.ProxyDeploymentManagerService.submitJsonWithRetry
WARNING: Failed (https://localhost:8543/deploymanager/auth/login)(attempt 3 of 3): Failed with HTTP error code : 404
additional information : <!doctype html><html lang="en"><head><title>HTTP Status 404 ▒~@~S Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 ▒~@~S Not Found</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Message</b> The requested resource [/deploymanager/auth/login] is not available</p><p><b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p><hr class="line" /><h3>Apache Tomcat</h3></body></html>
08/25-16:53:07.00036 [RMI TCP Connection(122)-192.168.33.10#1523] com.avamar.mc.util.MCException.logException
WARNING: com.avamar.asn.service.ServiceException: Failed with HTTP error code : 404
additional information : <!doctype html><html lang="en"><head><title>HTTP Status 404 ▒~@~S Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 ▒~@~S Not Found</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Message</b> The requested resource [/deploymanager/auth/login] is not available</p><p><b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p><hr class="line" /><h3>Apache Tomcat</h3></body></html>
at com.avamar.mc.pdm.ProxyDeploymentManagerService.login(ProxyDeploymentManagerService.java:577)
at com.avamar.mc.pdm.ProxyDeploymentManagerService.getAllTasks(ProxyDeploymentManagerService.java:377)
at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:61)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at com.sun.proxy.$Proxy96.getAllTasks(Unknown Source)
at com.avamar.mc.pdm.ProxyDeploymentManagerServiceContext.getAllTasks(ProxyDeploymentManagerServiceContext.java:69)
at sun.reflect.GeneratedMethodAccessor94.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
at sun.rmi.transport.Transport$1.run(Unknown Source)
at sun.rmi.transport.Transport$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Unknown Source)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
Confirm the 404 error at the following endpoint:
admin@ave:/usr/local/avamar/var/mc/server_log/>: curl -kv https://localhost:8543/deploymanager/auth/login
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8543 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=US; ST=CA; L=Irvine; OU=Dell EMC; O=Dell Technologies; CN=ave.lab
* start date: Jul 1 12:02:13 2022 GMT
* expire date: Jun 28 12:02:13 2032 GMT
* issuer: C=US; ST=CA; L=Irvine; OU=Dell EMC; O=Dell Technologies; CN=ave.lab
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /deploymanager/auth/login HTTP/1.1
> Host: localhost:8543
> User-Agent: curl/7.60.0
> Accept: */*
>
< HTTP/1.1 404
< Content-Type: text/html;charset=utf-8
< Content-Language: en
< Content-Length: 776
< Date: Thu, 25 Aug 2022 14:54:36 GMT
< Connection: close
< Server: Avamar
<
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1):
<!doctype html><html lang="en"><head><title>HTTP Status 404 – Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 – Not Found</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Message</b> The requested resource [/deploymanager/auth/login] is not available</p><p><b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p><hr class="line" /><h3>Apache Tomcat</h3></body></html>
The snapshot manager endpoint shows the following exception stack:
root@ave:/usr/local/avamar/var/mc/server_data/prefs/#: curl -kv https://localhost:8543/snapshotmanager/about
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8543 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=US; ST=CA; L=Irvine; OU=Dell EMC; O=Dell Technologies; CN=ave.lab
* start date: Jul 1 12:02:13 2022 GMT
* expire date: Jun 28 12:02:13 2032 GMT
* issuer: C=US; ST=CA; L=Irvine; OU=Dell EMC; O=Dell Technologies; CN=ave.lab
* SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /snapshotmanager/about HTTP/1.1
> Host: localhost:8543
> User-Agent: curl/7.60.0
> Accept: */*
>
< HTTP/1.1 500
< Content-Type: text/html;charset=utf-8
< Content-Language: en
< Transfer-Encoding: chunked
< Date: Thu, 25 Aug 2022 15:04:43 GMT
< Connection: close
< Server: Avamar
<
<!doctype html><html lang="en"><head><title>HTTP Status 500 – Internal Server Error</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 500 – Internal Server Error</h1><hr class="line" /><p><b>Type</b> Exception Report</p><p><b>Message</b> Servlet.init() for servlet [snapshotmanager] threw exception</p><p><b>Description</b> The server encountered an unexpected condition that prevented it from fulfilling the request.</p><p><b>Exception</b></p><pre>javax.servlet.ServletException: Servlet.init() for servlet [snapshotmanager] threw exception
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:543)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
[...]
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Unknown Source)
</pre><p><b>Root Cause</b></p><pre>org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'snapshotManagerServerConfiguration': Unsatisfied dependency expressed through field 'snapshotRequestHandler'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'snapshotHandler': Unsatisfied dependency expressed through field 'snapshotTemplate'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'getRestTemplate' defined in com.emc.avamar.vmware.snapshot.config.server.SnapshotManagerServerConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.rabbit.core.RabbitTemplate]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'getRestTemplate' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'replySnapshotQueue' defined in com.emc.avamar.vmware.snapshot.config.server.SnapshotManagerServerConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.core.Queue]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'replySnapshotQueue' threw exception; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:598)
org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:90)
[...]
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Unknown Source)
</pre><p><b>Root Cause</b></p><pre>org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'snapshotHandler': Unsatisfied dependency expressed through field 'snapshotTemplate'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'getRestTemplate' defined in com.emc.avamar.vmware.snapshot.config.server.SnapshotManagerServerConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.rabbit.core.RabbitTemplate]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'getRestTemplate' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'replySnapshotQueue' defined in com.emc.avamar.vmware.snapshot.config.server.SnapshotManagerServerConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.core.Queue]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'replySnapshotQueue' threw exception; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:598)
[...]
java.lang.Thread.run(Unknown Source)
</pre><p><b>Root Cause</b></p><pre>org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'getRestTemplate' defined in com.emc.avamar.vmware.snapshot.config.server.SnapshotManagerServerConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.rabbit.core.RabbitTemplate]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'getRestTemplate' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'replySnapshotQueue' defined in com.emc.avamar.vmware.snapshot.config.server.SnapshotManagerServerConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.core.Queue]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'replySnapshotQueue' threw exception; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:627)
[...]
java.lang.Thread.run(Unknown Source)
</pre><p><b>Root Cause</b></p><pre>org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.rabbit.core.RabbitTemplate]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'getRestTemplate' threw exception; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'replySnapshotQueue' defined in com.emc.avamar.vmware.snapshot.config.server.SnapshotManagerServerConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.core.Queue]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'replySnapshotQueue' threw exception; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
[...]
java.lang.Thread.run(Unknown Source)
</pre><p><b>Root Cause</b></p><pre>org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'replySnapshotQueue' defined in com.emc.avamar.vmware.snapshot.config.server.SnapshotManagerServerConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.core.Queue]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'replySnapshotQueue' threw exception; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:627)
org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:456)
[...]
java.lang.Thread.run(Unknown Source)
</pre><p><b>Root Cause</b></p><pre>org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.amqp.core.Queue]: Circular reference involving containing bean 'snapshotManagerServerConfiguration' - consider declaring the factory method as static for independence from its containing instance. Factory method 'replySnapshotQueue' threw exception; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
[...]
java.lang.Thread.run(Unknown Source)
</pre><p><b>Root Cause</b></p><pre>org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:61)
[...]
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Unknown Source)
</pre><p><b>Root Cause</b></p><pre>com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:339)
com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:647)
com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:694)
[...]
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Unknown Source)
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, Client hello (1):
</pre><p><b>Note</b> The full stack trace of the root cause is available in the server logs.</p><hr class="line" /><h3>Apache Tomcat</h3></body></html>
From the output above, collect the following snippet mentioned previously:
threw exception; nested exception is org.springframework.amqp.AmqpAuthenticationException: com.rabbitmq.client.AuthenticationFailureException: ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile.
Resolution
Avamar versions 19.x onwards:
1. Take a backup of mcserver.xml and update the RabbitMQ password in mcserver.xml to the default 'Avamar_Rabbit' and edit the file:
cp -p /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml /usr/local/avamar/var/mc/server_data/prefs/x-mcserver.xml
vi /usr/local/avamar/var/mc/server_data/prefs/mcserver.xml
2. Change the <entry key="rabbitmqAP" to the default password 'Avamar_Rabbit'
<entry key="rabbitmqAP" value="Avamar_Rabbit" />
3. Restart the Management Console Service (MCS), as admin user:
mcserver.sh --restart --force
4. Re-create the RabbitMQ User/Permissions/Password/vHost as root user, and use the following steps to do so automatically.
service rabbitmq-server stop
cd /data01/avamar/var/rabbitmq/mnesia
mkdir /data01/rabbit_mnesia_backup
mv rabbit@localhost* /data01/rabbit_mnesia_backup/
ls -ltr /data01/rabbit_mnesia_backup/
service rabbitmq-server start
msgbrokerctl.pl --remove
msgbrokerctl.pl --stop
msgbrokerctl.pl --start
5. Restart Tomcat as root user.
emwebapp.sh --restart
6. Test PDM recommendation.