Start a Conversation

Unsolved

This post is more than 5 years old

1338

March 19th, 2017 20:00

Gateway error while call any api.

Hi all.

I install scaleio version:

version.date=2016-12-23 09:33

version.build_major=2

version.build_minor=0

version.build_revision=12000

version.build_number=122

I exe call to api https:// :443/api/types/Sds/instances

{u'errorCode': 1,

u'httpStatusCode': 500,

u'message': u'java.util.concurrent.TimeoutException: Could not achieve full QueryPoll response in 10000 requests'}

2017-03-20 09:55:12,756 [http-nio-443-exec-8] ERROR c.e.s.s.w.c.ScaleIOController - Got an exception in handleException

com.emc.s3g.scaleio.common.gateway.ScaleIORemoteRuntimeException: java.util.concurrent.TimeoutException: Could not achieve full QueryPoll response in 10000 requests

        at com.emc.s3g.scaleio.repository.BaseModelBasedRepository.updateModel(BaseModelBasedRepository.java:111) ~[repository-1.0-SNAPSHOT.jar:na]

        at com.emc.s3g.scaleio.repository.BaseModelBasedRepository.query(BaseModelBasedRepository.java:39) ~[repository-1.0-SNAPSHOT.jar:na]

        at com.emc.s3g.scaleio.repository.BaseRepository.getGenObjects(BaseRepository.java:1049) ~[repository-1.0-SNAPSHOT.jar:na]

        at com.emc.s3g.scaleio.repository.BaseRepository.findAll(BaseRepository.java:197) ~[repository-1.0-SNAPSHOT.jar:na]

        at com.emc.s3g.scaleio.web.controller.ScaleIOController.getInstances(ScaleIOController.java:49) ~[ScaleIOController.class:na]

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_45]

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_45]

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_45]

        at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_45]

        at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:215) ~[ams-1.0-SNAPSHOT.jar:na]

        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132) ~[ams-1.0-SNAPSHOT.jar:na]

        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104) ~[spring-webmvc-3.2.9.RELEASE.jar:3.2.9.RELEASE]

        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:745) ~[spring-webmvc-3.2.9.RELEASE.jar:3.2.9.RELEASE]

        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:685) ~[spring-webmvc-3.2.9.RELEASE.jar:3.2.9.RELEASE]

        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80) ~[spring-webmvc-3.2.9.RELEASE.jar:3.2.9.RELEASE]

        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:919) ~[spring-webmvc-3.2.9.RELEASE.jar:3.2.9.RELEASE]

        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:851) ~[spring-webmvc-3.2.9.RELEASE.jar:3.2.9.RELEASE]

        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:953) [spring-webmvc-3.2.9.RELEASE.jar:3.2.9.RELEASE]

        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:844) [spring-webmvc-3.2.9.RELEASE.jar:3.2.9.RELEASE]

        at javax.servlet.http.HttpServlet.service(HttpServlet.java:620) [servlet-api.jar:na]

        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:829) [spring-webmvc-3.2.9.RELEASE.jar:3.2.9.RELEASE]

        at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) [servlet-api.jar:na]

        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) [catalina.jar:7.0.57]

        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57]

        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat7-websocket.jar:7.0.57]

        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.57]

        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57]

        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) [spring-security-web-3.2.4.RELEASE.jar:3.

2.4.RELEASE]

        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:154) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:50) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106) [ams-1.0-SNAPSHOT.jar:na]

        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.access.channel.ChannelProcessingFilter.doFilter(ChannelProcessingFilter.java:144) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) [spring-security-web-3.2.4.RELEASE.jar:3.2.4.RELEASE]

  at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) [ams-1.0-SNAPSHOT.jar:na]

        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) [ams-1.0-SNAPSHOT.jar:na]

        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) [catalina.jar:7.0.57]

        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) [catalina.jar:7.0.57]

        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) [catalina.jar:7.0.57]

        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) [catalina.jar:7.0.57]

        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:612) [catalina.jar:7.0.57]

        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170) [catalina.jar:7.0.57]

        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) [catalina.jar:7.0.57]

        at ch.qos.logback.access.tomcat.LogbackValve.invoke(LogbackValve.java:178) [logback-access-1.0.13.jar:na]

        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) [catalina.jar:7.0.57]

        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421) [catalina.jar:7.0.57]

        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070) [tomcat-coyote.jar:7.0.57]

        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611) [tomcat-coyote.jar:7.0.57]

        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1736) [tomcat-coyote.jar:7.0.57]

        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1695) [tomcat-coyote.jar:7.0.57]

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_45]

        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-coyote.jar:7.0.57]

        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]

Caused by: java.util.concurrent.TimeoutException: Could not achieve full QueryPoll response in 10000 requests

        at com.emc.ecs.model.QueryPollMngr.processChunkedUpdate(QueryPollMngr.java:144) ~[ams-1.0-SNAPSHOT.jar:na]

        at com.emc.s3g.scaleio.repository.BaseModelBasedRepository.updateModel(BaseModelBasedRepository.java:78) ~[repository-1.0-SNAPSHOT.jar:na]

        ... 67 common frames omitted

306 Posts

March 20th, 2017 05:00

Hi hoanhdo,

Please try to restart the GW service and if that not help, try to switch the MDM ownership and run it one more time.

Thank you,

Pawel

March 20th, 2017 07:00

Can you please provide some more information regarding the setup?

What OS are you deploying ScaleIO on? is that ESXi environment or physical Linux Nodes?

Is the GW installed on the same node as one of the MDMs (in case of Linux)? In case using ESXi enviroment, is the GW SVM deployed on the same ESXi node as one of the MDM SVM?

In case the GW is deployed on a different host, can you please make sure it has access to the same subnets as the MDMs?

15 Posts

March 20th, 2017 18:00

Hi all.

I restart gateway many times. It still error.

My scaleio deployed in physical linux , same node with mdm and sds. It have 1 network.

Uname -a

Linux scaleio-02 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Redhat-realse

CentOS Linux release 7.3.1611 (Core)

Thanks.

March 21st, 2017 05:00

CentOS 7.3 i guess is not supported. Please install CentOS 7.2 and check.

March 21st, 2017 05:00

What is the java version installed?

15 Posts

March 21st, 2017 18:00

I installed java 1.8

java version "1.8.0_45"

Java(TM) SE Runtime Environment (build 1.8.0_45-b14)

Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

I will install centos 7.2 and check.

Thanks

March 23rd, 2017 05:00

Is your issue resolved after installing CentOS 7.2?

No Events found!

Top