UNSOLVED
C
codemonkey3
9 Posts
0
2480
June 9th, 2015 08:00
Web Services API java.util.ConcurrentModificationException
Hello all, I'm running into an exception periodically using the WATCH4NET web service api. The top level exception reports "can't get valid user", however the cause the of the exception is a current modification on array list sorting. I happens quite often, seems like this area of code is not thread safe. There are multiple threads/users accessing the web services api.
Has anyone seen/experienced this issue? Any feed back would be great!
Version: EMC M&R v6.5u1 - 54340
SEVERE
javax.xml.ws.soap.SOAPFaultException: can't get valid user
at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178)
at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:125)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:108)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:78)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:135)
at com.sun.proxy.$Proxy83.getDistinctPropertyValues(Unknown Source)
at com.cirba.emcsrm.EmcSrmClient.getDistinctPropertyValues(EmcSrmClient.java:207)
at com.cirba.emcsrm.EmcSrmClient.getPropertyValue(EmcSrmClient.java:466)
at com.cirba.emcsrm.EmcSrmRequestAdministrator.getConfigPropertyValue(EmcSrmRequestAdministrator.java:1221)
at com.cirba.emcsrm.EmcSrmRequestAdministrator.addConfigTableRows(EmcSrmRequestAdministrator.java:969)
at com.cirba.emcsrm.EmcSrmRequestAdministrator.processTableRows(EmcSrmRequestAdministrator.java:926)
at com.cirba.emcsrm.EmcSrmRequestAdministrator.processObject(EmcSrmRequestAdministrator.java:873)
at com.cirba.emcsrm.EmcSrmRequestAdministrator.executeEntity(EmcSrmRequestAdministrator.java:811)
at com.cirba.emcsrm.EmcSrmRequestAdministrator$Entity.run(EmcSrmRequestAdministrator.java:790)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.SecurityException: can't get valid user
at com.watch4net.apg.v2.remote.security.ApplicationUserMapperProvider.getUser(ApplicationUserMapperProvider.java:61)
at com.watch4net.apg.v2.remote.impl.DatabaseAccessor.getUser(DatabaseAccessor.java:110)
at com.watch4net.apg.v2.remote.impl.DatabaseAccessor.getDistinctPropertyValues(DatabaseAccessor.java:168)
at sun.reflect.GeneratedMethodAccessor298.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.sun.xml.ws.api.server.InstanceResolver$1.invoke(InstanceResolver.java:246)
at com.sun.xml.ws.server.InvokerTube$2.invoke(InvokerTube.java:146)
at com.sun.xml.ws.server.sei.EndpointMethodHandler.invoke(EndpointMethodHandler.java:258)
at com.sun.xml.ws.server.sei.SEIInvokerTube.processRequest(SEIInvokerTube.java:93)
at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:598)
at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:557)
at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:542)
at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:439)
at com.sun.xml.ws.server.WSEndpointImpl$2.process(WSEndpointImpl.java:243)
at com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter.java:444)
at com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:244)
at com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapter.java:135)
at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.doGet(WSServletDelegate.java:129)
at com.sun.xml.ws.transport.http.servlet.WSServletDelegate.doPost(WSServletDelegate.java:160)
at com.sun.xml.ws.transport.http.servlet.WSServlet.doPost(WSServlet.java:75)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:643)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.watch4net.apg.logging.webapp.MDCPopulatingFilter.doFilter(MDCPopulatingFilter.java:76)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.watch4net.apg.v2.gui.tomcat.SSLDownloadHeaderFixFilter.doFilter(SourceFile:51)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:563)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615)
at org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:394)
at com.watch4net.apg.v2.gui.tomcat.SessionIdRegenValve.invoke(SourceFile:82)
at com.watch4net.apg.v2.gui.tomcat.LoggingValve.invoke(SourceFile:40)
at com.watch4net.apg.v2.gui.tomcat.EncodingFixValve.invoke(SourceFile:26)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:620)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
... 1 more
Caused by: java.util.ConcurrentModificationException
at java.util.ArrayList.sort(ArrayList.java:1456)
at java.util.Collections.sort(Collections.java:141)
at com.watch4net.apg.v2.gui.tree.RealNode.sortRealChildren(RealNode.java:192)
at com.watch4net.apg.v2.gui.tree.TreeUtils.sortRealChildren(TreeUtils.java:86)
at com.watch4net.apg.v2.gui.tree.TreeUtils.sortAndResolve(TreeUtils.java:69)
at com.watch4net.apg.v2.gui.user.RootNode.initialize(RootNode.java:193)
at com.watch4net.apg.v2.gui.master.MasterUtils.loadUser(MasterUtils.java:279)
at com.watch4net.apg.v2.gui.master.MasterUtils.loadUser(MasterUtils.java:236)
at com.watch4net.apg.v2.gui.master.MasterUtils.loadUser(MasterUtils.java:190)
at com.watch4net.apg.v2.remote.security.MasterDatabaseAccessorMapper.loadUser(MasterDatabaseAccessorMapper.java:85)
at com.watch4net.apg.v2.remote.security.MasterDatabaseAccessorMapper.getApplicationUser(MasterDatabaseAccessorMapper.java:51)
at com.watch4net.apg.v2.remote.security.ApplicationUserMapperProvider.getUser(ApplicationUserMapperProvider.java:59)
... 46 more
PaulORourke
170 Posts
1808
0
Posted June 18th, 2015 00:00
Hi codemonkey,
Found a similar error and the fix was to use:
apg.ws.usermapper=com.watch4net.apg.v2.remote.security.UnrestrictedUsageMapper
in
APG/Web-Applications/APG-Web-Service/APG-WS/conf/common.properties
Can you please try this?
Thanks,Kind Regards,
Paul O'Rourke
codemonkey3
9 Posts
1808
0
Posted June 18th, 2015 03:00
Thank you so much Paul for the response!
I've applied the change, I'll let run for while to see if resolves the issue and let you know
Again thank you!
The following configuration change will disable all user security check on the webservice. It is not recommended on production environment, unless you put in place external security mesure (firewalls, ...)
The error "java.util.ConcurrentModificationException" really indicate that you have hit a bug. It could be related to too many API calls, but anyway should be handled better with query queuing.
I recommend a SR ticket for this.
Have a nice day
codemonkey3
9 Posts
1808
0
Posted June 19th, 2015 04:00
Paul's suggested fix does seem to resolve the issue, however as you've pointed out not a desirable change in a production environment.
Indeed, the java.util.ConcurrentModificationException indicates we've ran into to a bug, moreover this indicates threading issues where a block(s) of code are not guarded correctly i.e arraylist member of runable class accessed by multiple threads of which modifications to the data are happening.
How would I go about creating a bug for this?
Thank you both very much
When you log on the support site, there is a section to create a ticket. Just make sure to select the SRM product.
If you want, I can do it myself internally, but having this from a customer/partner really help push the fix asap. It will also give you the chance to get clear ETA and know about which version get the fix.
Have a nice day
codemonkey3
9 Posts
1808
0
Posted June 17th, 2015 11:00
Correction on the version: 3.5.1.0.117
If anyone could shed some light on this I'd appreciate it!