Avamar: Der DPE-vcpsrv-Service ist aufgrund einer fehlgeschlagenen Verbindung mit externem rabbitmq ausgefallen.

Der vcpsrv-Dienst auf dem DPE kann mit dem folgenden Fehler nicht gestartet werden:
 

2023-01-25 09:56:48,601 [main] INFO  (StartupInfoLogger.java:48) - Starting Application v19.7.0.90 on  with PID 6 (/opt/vcp/vcpsrv/vcp-server.jar started by root in /opt/vcp/vcpsrv)
2023-01-25 09:56:48,610 [main] INFO  (SpringApplication.java:593) - No active profile set, falling back to default profiles: default
2023-01-25 09:56:51,109 [main] INFO  (CstServiceManager.java:147) - Creating Service Manager
2023-01-25 09:56:52,045 [main] INFO  (CstServiceManager.java:161) - Done: Created Service manager
2023-01-25 09:56:52,046 [main] INFO  (VcpCstService.java:529) - Created the CstServiceManager successfully
2023-01-25 09:56:52,049 [main] INFO  (CstLBStore.java:292) - Creating LockBox Store
2023-01-25 09:56:52,053 [main] INFO  (CstLBStore.java:298) - Done: Created LockBox Store
2023-01-25 09:56:52,054 [main] INFO  (VcpCstService.java:542) - Created the LBStore successfully
2023-01-25 09:56:53,160 [main] INFO  (CredentialedDataSource.java:122) - Postgres Version: PostgreSQL 9.6.13 on x86_64-pc-linux-gnu, compiled by gcc (SUSE Linux) 4.8.5, 64-bit
2023-01-25 09:56:53,227 [main] INFO  (Migration.java:44) - Starting Database Validation and Migration...
2023-01-25 09:56:53,240 [main] INFO  (Migration.java:53) - Found table 'schema_version', no need to initialize.
2023-01-25 09:56:54,637 [main] WARN  (SequenceGenerator.java:87) - HHH90000014: Found use of deprecated [org.hibernate.id.SequenceGenerator] sequence-based id generator; use org.hibernate.id.enhanced.SequenceStyleGenerator instead.
 See Hibernate Domain Model Mapping Guide for details.
2023-01-25 09:56:54,646 [main] WARN  (SequenceGenerator.java:87) - HHH90000014: Found use of deprecated [org.hibernate.id.SequenceGenerator] sequence-based id generator; use org.hibernate.id.enhanced.SequenceStyleGenerator instead.
 See Hibernate Domain Model Mapping Guide for details.
2023-01-25 09:56:57,631 [main] INFO  (VcdClient.java:715) - Watchdog: vCloud session created: 1674640617631
2023-01-25 09:56:57,633 [main] INFO  (VcdClient.java:282) - VCP-Service Version: 19.7.0.90
2023-01-25 09:56:57,714 [main] WARN  (AbstractApplicationContext.java:551) - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amqpConfiguration': Invocation of init method failed; nested exception is org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
2023-01-25 09:56:57,715 [main] INFO  (VcdClient.java:287) - Destroying VcdClient...
2023-01-25 09:56:57,728 [main] ERROR (SpringApplication.java:771) - Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amqpConfiguration': Invocation of init method failed; nested exception is org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:137) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:409) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1626) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) ~[spring-context-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) ~[spring-context-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693) ~[spring-boot-1.5.13.RELEASE.jar!/:1.5.13.RELEASE]
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360) ~[spring-boot-1.5.13.RELEASE.jar!/:1.5.13.RELEASE]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:303) ~[spring-boot-1.5.13.RELEASE.jar!/:1.5.13.RELEASE]
        at com.emc.vcp.Application.main(Application.java:51) ~[classes!/:19.7.0.90]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_301]
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_301]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:1.8.0_301]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[?:1.8.0_301]
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) ~[vcp-server.jar:19.7.0.90]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) ~[vcp-server.jar:19.7.0.90]
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) ~[vcp-server.jar:19.7.0.90]
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) ~[vcp-server.jar:19.7.0.90]
Caused by: org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
        at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71) ~[spring-rabbit-1.7.7.RELEASE.jar!/:?]
        at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:376) ~[spring-rabbit-1.7.7.RELEASE.jar!/:?]
.
.
.
.
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketOutputStream.socketWrite(Unknown Source) ~[?:1.8.0_301]
        at java.net.SocketOutputStream.write(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketOutputRecord.flush(Unknown Source) ~[?:1.8.0_301]
.
.
.
.
.       at sun.security.ssl.HandshakeOutStream.flush(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.ECDHClientKeyExchange$ECDHEClientKeyExchangeProducer.produce(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.ClientKeyExchange$ClientKeyExchangeProducer.produce(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLHandshake.produce(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.ServerHelloDone$ServerHelloDoneConsumer.consume(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLHandshake.consume(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.HandshakeContext.dispatch(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.HandshakeContext.dispatch(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.TransportContext.dispatch(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLTransport.decode(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.decode(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.ensureNegotiated(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl.access$200(Unknown Source) ~[?:1.8.0_301]
        at sun.security.ssl.SSLSocketImpl$AppOutputStream.write(Unknown Source) ~[?:1.8.0_301]
        at java.io.BufferedOutputStream.flushBuffer(Unknown Source) ~[?:1.8.0_301]
        at java.io.BufferedOutputStream.flush(Unknown Source) ~[?:1.8.0_301]
        at java.io.DataOutputStream.flush(Unknown Source) ~[?:1.8.0_301]
        at com.rabbitmq.client.impl.SocketFrameHandler.sendHeader(SocketFrameHandler.java:147) ~[amqp-client-4.0.3.jar!/:4.0.3]
.
.
.
.
2023-01-25 09:58:11,558 [main] INFO  (VcdClient.java:282) - VCP-Service Version: 19.7.0.90
2023-01-25 09:58:11,632 [main] WARN  (AbstractApplicationContext.java:551) - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amqpConfiguration': Invocation of init method failed; nested exception is org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
2023-01-25 09:58:11,633 [main] INFO  (VcdClient.java:287) - Destroying VcdClient...
2023-01-25 09:58:11,644 [main] ERROR (SpringApplication.java:771) - Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'amqpConfiguration': Invocation of init method failed; nested exception is org.springframework.amqp.AmqpIOException: javax.net.ssl.SSLException: readHandshakeRecord
        at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:137) ~[spring-beans-4.3.17.RELEASE.jar!/:4.3.17.RELEASE]

Der VCP RabbitMQ TLS-Handshake versucht, eine gegenseitige TLS-Authentifizierung durchzuführen, und sein Handshake wird von RabbitMQ zurückgesetzt, nachdem vcpsrv das DPE-Zertifikat sendet. 

Der externe RabbitMQ-Server muss angewiesen werden, dem DPE-Stammzertifikat zu vertrauen, das zum Signieren von vcpsrv-Zertifikaten (und vcpbg)-Zertifikaten verwendet wird, damit der RabbitMQ-Datenverkehr gegenseitig tls authentifiziert werden kann.

Führen Sie die folgenden Schritte aus, um folgendes zu erreichen:

1. Verwenden Sie keytool, um die DPE-Stammzertifizierungsstelle (und das dpem-Zertifikat) abzurufen.

Auf DPE-Node: 

keytool -printcert -sslserver localhost:9000 -rfc | sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' | awk '/-----BEGIN CERTIFICATE-----/{i++}{print > "cert"i".pem"}'

Dadurch sollten eine cert1.pem- und cert2.pem-Datei erstellt werden. 
cert2.pem sollte dpe root ca sein.

2. Fügen Sie eine weitere CAfile aus Schritt A zur rabbitmq.config des Kunden auf dem ersten rabbitmq-Node hinzu.

Auf Rabbitmq-Node:
Bearbeiten Sie die Datei unter /etc/rabbitmq/ssl/cacert.pem und fügen Sie den Inhalt von /path/to/cert2.pem am Ende der Datei hinzu:

{cacertfile, "/etc/rabbitmq/ssl/cacert.pem"}

Starten Sie nach dieser Änderung RabbitMQ neu, damit die Änderungen wirksam werden.

3. Wiederholen Sie Schritt 2 für andere rabbitmq-Nodes.