Avamar: Il servizio DPE vcpsrv è inattivo a causa di una connessione non riuscita con rabbitmq esterno.

Il servizio vcpsrv su DPE non si avvia con il seguente errore:
 

2023-01-25 09:56:48,601 [main] INFO  (StartupInfoLogger.java:48) - Starting Application v19.7.0.90 on  with PID 6 (/opt/vcp/vcpsrv/vcp-server.jar started by root in /opt/vcp/vcpsrv)
2023-01-25 09:56:48,610 [main] INFO  (SpringApplication.java:593) - No active profile set, falling back to default profiles: default
2023-01-25 09:56:51,109 [main] INFO  (CstServiceManager.java:147) - Creating Service Manager
2023-01-25 09:56:52,045 [main] INFO  (CstServiceManager.java:161) - Done: Created Service manager
2023-01-25 09:56:52,046 [main] INFO  (VcpCstService.java:529) - Created the CstServiceManager successfully
2023-01-25 09:56:52,049 [main] INFO  (CstLBStore.java:292) - Creating LockBox Store
2023-01-25 09:56:52,053 [main] INFO  (CstLBStore.java:298) - Done: Created LockBox Store
2023-01-25 09:56:52,054 [main] INFO  (VcpCstService.java:542) - Created the LBStore successfully
2023-01-25 09:56:53,160 [main] INFO  (CredentialedDataSource.java:122) - Postgres Version: PostgreSQL 9.6.13 on x86_64-pc-linux-gnu, compiled by gcc (SUSE Linux) 4.8.5, 64-bit
2023-01-25 09:56:53,227 [main] INFO  (Migration.java:44) - Starting Database Validation and Migration...
2023-01-25 09:56:53,240 [main] INFO  (Migration.java:53) - Found table 'schema_version', no need to initialize.
2023-01-25 09:56:54,637 [main] WARN  (SequenceGenerator.java:87) - HHH90000014: Found use of deprecated [org.hibernate.id.SequenceGenerator] sequence-based id generator; use org.hibernate.id.enhanced.SequenceStyleGenerator instead.
 See Hibernate Domain Model Mapping Guide for details.
2023-01-25 09:56:54,646 [main] WARN  (SequenceGenerator.java:87) - HHH90000014: Found use of deprecated [org.hibernate.id.SequenceGenerator] sequence-based id generator; use org.hibernate.id.enhanced.SequenceStyleGenerator instead.
 See Hibernate Domain Model Mapping Guide for details.
2023-01-25 09:56:57,631 [main] INFO  (VcdClient.java:715) - Watchdog: vCloud session created: 1674640617631
2023-01-25 09:56:57,633 [main] INFO  (VcdClient.java:282) - VCP-Service Version: 19.7.0.90
2023-01-25 09:56:57,714 [main] WARN  (AbstractApplicationContext.java:551) - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amqpConfiguration': Invocation of init method failed; nested exception is org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
2023-01-25 09:56:57,715 [main] INFO  (VcdClient.java:287) - Destroying VcdClient...
2023-01-25 09:56:57,728 [main] ERROR (SpringApplication.java:771) - Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amqpConfiguration': Invocation of init method failed; nested exception is org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:137) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:409) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1626) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) ~[spring-context-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) ~[spring-context-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693) ~[spring-boot-1.5.13.RELEASE.jar!/:1.5.13.RELEASE]
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360) ~[spring-boot-1.5.13.RELEASE.jar!/:1.5.13.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) ~[spring-boot-1.5.13.RELEASE.jar!/:1.5.13.RELEASE]
        at com.emc.vcp.Application.main(Application.java:51) ~[classes!/:19.7.0.90]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_301]
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_301]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_301]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[?:1.8.0_301]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) ~[vcp-server.jar:19.7.0.90]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) ~[vcp-server.jar:19.7.0.90]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) ~[vcp-server.jar:19.7.0.90]
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) ~[vcp-server.jar:19.7.0.90]
Caused by: org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
        at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71) ~[spring-rabbit-1.7.7.RELEASE.jar!/:?]
        at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:376) ~[spring-rabbit-1.7.7.RELEASE.jar!/:?]
.
.
.
.
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[?:1.8.0_301]
        at java.net.SocketOutputStream.write(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketOutputRecord.flush(Unknown Source) ~[?:1.8.0_301]
.
.
.
.
.       at sun.security.ssl.HandshakeOutStream.flush(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.ECDHClientKeyExchange$ECDHEClientKeyExchangeProducer.produce(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.ClientKeyExchange$ClientKeyExchangeProducer.produce(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLHandshake.produce(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.ServerHelloDone$ServerHelloDoneConsumer.consume(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLHandshake.consume(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.HandshakeContext.dispatch(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.HandshakeContext.dispatch(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.TransportContext.dispatch(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLTransport.decode(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.decode(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.ensureNegotiated(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.access$200(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl$AppOutputStream.write(Unknown Source) ~[?:1.8.0_301]
        at java.io.BufferedOutputStream.flushBuffer(Unknown Source) ~[?:1.8.0_301]
        at java.io.BufferedOutputStream.flush(Unknown Source) ~[?:1.8.0_301]
        at java.io.DataOutputStream.flush(Unknown Source) ~[?:1.8.0_301]
        at com.rabbitmq.client.impl.SocketFrameHandler.sendHeader(SocketFrameHandler.java:147) ~[amqp-client-4.0.3.jar!/:4.0.3]
.
.
.
.
2023-01-25 09:58:11,558 [main] INFO  (VcdClient.java:282) - VCP-Service Version: 19.7.0.90
2023-01-25 09:58:11,632 [main] WARN  (AbstractApplicationContext.java:551) - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amqpConfiguration': Invocation of init method failed; nested exception is org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
2023-01-25 09:58:11,633 [main] INFO  (VcdClient.java:287) - Destroying VcdClient...
2023-01-25 09:58:11,644 [main] ERROR (SpringApplication.java:771) - Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amqpConfiguration': Invocation of init method failed; nested exception is org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:137) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]

L'handshake VCP RabbitMQ TLS sta tentando di eseguire un'autenticazione tls reciproca e l'handshake viene reimpostata da RabbitMQ dopo che vcpsrv invia il certificato DPE. 

Al server RabbitMQ esterno deve essere detto di considerare attendibile il certificato radice DPE utilizzato per firmare i certificati vcpsrv (e vcpbg) per consentire l'autenticazione reciproca del traffico RabbitMQ.

Attenersi alla procedura riportata di seguito per ottenere questo risultato:

1- Utilizzare keytool per recuperare la CA root DPE (e il certificato dpem).

Sul nodo DPE: 

keytool -printcert -sslserver localhost:9000 -rfc | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' | awk '/-----BEGIN CERTIFICATE-----/{i++}{print > "cert"i".pem"}'

In questo modo viene creato un file cert1.pem e cert2.pem. 
cert2.pem deve essere ca radice DPE.

2- Aggiungere un altro CAfile dal passaggio A al file rabbitmq.config del cliente sul primo nodo rabbitmq.

Nel nodo Rabbitmq:
modificare il file in /etc/rabbitmq/ssl/cacert.pem e aggiungere il contenuto di /path/to/cert2.pem alla fine del file:

{cacertfile, "/etc/rabbitmq/ssl/cacert.pem"}

Dopo questa modifica, riavviare RabbitMQ per rendere effettive le modifiche.

3 - Ripetere il passaggio 2 per altri nodi rabbitmq.