Vipr SRM | Watch4net : Additional backend is listening, and filled with Stream Closed errors

Summary: Additional Backend is running, but Collectors are not able push data due connection refusal's. Additonal backend is filled with Stream Closed errors,

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms



Various reports are missing Metrics, Properties. The Load Balancer a nd Arbiter are failing to  to push data to the Additional Backends, even though they show as started. further investigation into the Additional Backend show the following error in the logs:

WARNING -- [2017-01-10 15:16:15 EST] -- AbstractCollector::flushNext(): Error flushing Connector-Backend3 com.watch4net.apg.v2.collector.PipeError: Can't write to backend2.host.local:2300 at com.watch4net.apg.v2.collector.plugins.SocketConnector.sendBuffer(SocketConnector.java:441) at com.watch4net.apg.v2.collector.plugins.SocketConnector.commit(SocketConnector.java:409) at com.watch4net.apg.v2.collector.plugins.SocketConnector.flushData(SocketConnector.java:452) at com.watch4net.apg.v2.collector.plugins.loadbalancer.decision.chain.AbstractEndPointElement.flushData(AbstractEndPointElement.java:63) at com.watch4net.apg.v2.collector.AbstractCollector.flushNext(AbstractCollector.java:85) at com.watch4net.apg.v2.collector.plugins.FailOverFilter.access$1400(FailOverFilter.java:53) at com.watch4net.apg.v2.collector.plugins.FailOverFilter$CheckNextComponentAvailabilityTimerTask.run(FailOverFilter.java:906) at java.util.TimerThread.mainLoop(Timer.java:555) at java.util.TimerThread.run(Timer.java:505) Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at com.watch4net.apg.v2.collector.plugins.SocketConnector.connect(SocketConnector.java:562) at com.watch4net.apg.v2.collector.plugins.SocketConnector.ensureConnected(SocketConnector.java:595) at com.watch4net.apg.v2.collector.plugins.SocketConnector.sendBuffer(SocketConnector.java:424)

Cause

Additional Backend is currently not listening, and is filled with the following errors,
 

WARNING -- [2017-01-03 10:31:28 EST] -- StorageGroup::handleRawData(): Unable to write raw value variable 'x.x.x.x.x.x.x.x.x'! java.io.IOException: Stream Closed

Resolution

Restarting the Additional Backends host allowed the services to come up but in this Particular case, the customer was unaware that ABE had hit 100% File capacity, and some operations were no longer running as a result. The host would need to have been cleaned up or more space added.

Additional Information

Troubleshooting steps included:

running "netstat -anp | grep 2..." to verify that Backend sockets are listening on 2100, 2200, 2300, 2400 Host3:~ # netstat -anp | grep -i :2... tcp 0 0 0.0.0.0:2100 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:2101 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:2200 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:2201 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:2300 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:2301 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:2400 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:2401 0.0.0.0:* LISTEN tcp 0 0 0.0.0.0:2050 0.0.0.0:* LISTEN




Affected Products

SRM

Products

SRM
Article Properties
Article Number: 000055191
Article Type: Solution
Last Modified: 11 Oct 2024
Version:  3
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.