ViPR SRM: El módulo de alertas se bloquea
Summary: El módulo de alertas se bloquea en SRM
Symptoms
Se descubrió que había más de 900 conexiones al puerto 2013 (algunas dinámicas) que estaban en diferentes estados:WARNING [2017-04-10 03:49:42 EDT] RawValueDecoder::decode(): Invalid raw value rejectedINFO [2017-04-10 03:49:42 EDT] ChannelNegotiationProtocol::transform(): [id: 0xbff120bb, //xx.xxx.xxx.xxx:46555 => //xx.xxx.xxx.xxx:2010] Initializing channel with no capabilitiesINFO [2017-04-10 03:49:49 EDT] SocketSource::disconnect(): Dropping connection to /xx.xxx.xxx.xxx.INFO [2017-04-10 03:49:49 EDT] SocketSource::connect(): Accepted incoming connection from /xx.xxx.xxx.xxx.SRM version 4.0.1 vAppIssues found during webex:WARNING [2017-04-10 03:49:42 EDT] RawValueDecoder::decode(): Invalid raw value rejectedINFO [2017-04-10 03:49:42 EDT] ChannelNegotiationProtocol::transform(): [id: 0xbff120bb, //xx.xxx.xxx.xxx:46555 => //xx.xxx.xxx.xxx:2010] Initializing channel with no capabilitiesINFO [2017-04-10 03:49:49 EDT] SocketSource::disconnect(): Dropping connection to /xx.xxx.xxx.xxx.INFO [2017-04-10 03:49:49 EDT] SocketSource::connect(): Accepted incoming connection from /xx.xxx.xxx.xxx.WARNING [2017-04-10 03:49:49 EDT] SocketSource$DataReaderWorker::run(): An incoming event could not be processed: com.watch4net.events.common.serialization.SerializationException: Malformed occurrence keys in event: HEAD / HTTP/1.0WARNING [2017-04-10 03:49:49 EDT] SocketSource$DataReaderWorker::run(): An incoming event could not be processed: com.watch4net.events.common.serialization.SerializationException: Malformed definition keys in event: INFO [2017-04-10 03:49:50 EDT] BasicMessagesLoggingHandler::channelInactive(): [id: 0xbff120bb, /xx.xxx.xxx.xxx:46555 :> //xx.xxx.xxx.xxx:2010] Communication channel is now inactive/closedINFO [2017-04-10 03:49:50 EDT] BasicMessagesLoggingHandler::channelActive(): [id: 0xf28e525d, //xx.xxx.xxx.xxx:47531 => //xx.xxx.xxx.xxx:2010] Communication channel is now activeSEVERE [2017-04-10 03:49:50 EDT] ApplicationDataForwarder::unhandledExceptionCaught(): An unhandled error occured on channel [id: 0xf28e525d, //xx.xxx.xxx.xxx:47531 => //xx.xxx.xxx.xxx:2010]com.emc.watch4net.socket.communicator.handler.rawvalue.RawValueDecoder$InvalidRawValueExceptiAn incoming event could not be processed: com.watch4net.events.common.serialization.SerializationException: Malformed definition keys in event /xx.xxx.xxx.xxx pbe = Name:hostname.net Address: /xx.xxx.xxx.xxxWARNING [2017-04-12 05:14:26 EDT] SocketSource$DataReaderWorker::run(): An incoming event could not be processed: com.watch4net.events.common.serialization.SerializationException: Malformed occurence keys in event: [1]ClientHelloWARNING [2017-04-12 05:14:26 EDT] SocketSource$DataReaderWorker::run(): An incoming event could not be processed: com.watch4net.events.common.serialization.SerializationException: Malformed occurence keys in event: EndMessageINFO [2017-04-12 05:14:34 EDT] SocketSource::disconnect(): Dropping connection to 10.xxx.xxx.xxx.INFO [2017-04-12 05:14:34 EDT] SocketSource::connect(): Accepted incoming connection from 10.xxx.xxx.xxx.WARNING [2017-04-12 05:15:46 EDT] SocketSource$DataReaderWorker::run(): An incoming event could not be processed: com.watch4net.events.common.serialization.SerializationException: Malformed occurence keys in event: ________________________________________
Cause
bunit-group.csv no es válido
fe = xx.xxx.xxx.xxx Name: jxqpstgsrmfe01.onefiserv.net Address: xx.xxx.xxx.xxxOPEN CONNECTIONS:tcp 96848 0 xx.xxx.xxx.xxx:2013 xx.xxx.xxx.xxx:35817 ESTABLISHED off (0.00/0/0)tcp 96848 0 1xx.xxx.xxx.xxx:2013 xx.xxx.xxx.xxx:55989 ESTABLISHED off (0.00/0/0)tcp 96848 0 xx.xxx.xxx.xxx:2013 xx.xxx.xxx.xxx:60556 ESTABLISHED off (0.00/0/0)tcp 30713 0 xx.xxx.xxx.xxx:2013 xx.xxx.xxx.xxx:50174 CLOSE_WAIT off (0.00/0/0)tcp 43047 0 xx.xxx.xxx.xxx:2013 xx.xxx.xxx.xxx:38699 ESTABLISHED off (0.00/0/0)tcp 22574 0 xx.xxx.xxx.xxx:2013 xx.xxx.xxx.xxx:58961 CLOSE_WAIT off (0.00/0/0)tcp 98467 0 xx.xxx.xxx.xxx:2013 xx.xxx.xxx.xxx:43472 ESTABLISHED off (0.00/0/0)tcp 0 50945 1xx.xxx.xxx.xxx:50364 xx.xxx.xxx.xxx:2013 FIN_WAIT1 unkn-4 (2.26/0/0)Java Heap:apg 32467 1 5 Apr04 ? 12:02:19 /opt/APG/Java/Sun-JRE/8.0.102/bin/java -Xms256m -Xmx2048m -javaagent:/opt/APG/bin/.runtime/service/1.10u4/apg-bootstrap-agent.jar -Djava.rmi.server.hostname=jxqpstgsrmbe01.onefiserv.net -Djava.util.logging.config.file=conf/alerting.logging.properties -Dcom.watch4net.utils.jmx.agent.config.file=conf/w4n-agent.properties -Dcom.watch4net.utils.jmx.agent.host=jxqpstgsrmbe01.onefiserv.net -javaagent:lib/w4n-jmx-agent.jar -cp /opt/APG/bin/.runtime/service/1.10u4/apg-service-bootstrap.jar:lib/* com.watch4net.apg.module.plugin.service.Bootstrap com.watch4net.alerting.engine.AlertingEngine main start
Resolution
La solución alternativa proporcionada allí se ve así:
Agregar la solución alternativa a apg.properties fDe esta manera, la mayoría de las personas que /opt/APG/bin de back-end primario, back-end adicional y todos los recopiladores, si es posible, para saber de qué recopilador provienen los datos framelegnth es excedente, aunque el puntero está en localhost, pero es bueno agregarlo en cada host de recopilador. (Nota: Primero, haga un respaldo del archivo apg.properties).
Agregue la siguiente línea al final del archivo: /opt/APG/bin/apg.properties de cada host de SRM (BE primario, BE adicional y recopiladores)restricted.reader.line.size=50000
A continuación, reinicie los servicios de back-end de alertas.