Plataforma de gerenciamento do PowerFlex: Falha nas chamadas de API do PowerFlex Gateway devido ao espaço de heap do Java

Summary: As chamadas de API do OpenStack e de outras integrações semelhantes à PowerFlex Management Platform (PFMP) falham devido ao esgotamento do espaço de heap do Java nos pods do PowerFlex Gateway. Reiniciar os pods block-gateway afetados resolve o problema temporariamente. ...

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

  • As chamadas de API para o PowerFlex Manager (PFMP) falham intermitentemente, afetando o OpenStack e outras integrações.
  • block-gateway-0 Relatórios do pod OutOfMemoryErrore as solicitações são redirecionadas para block-gateway-1. 
    06:30:21.635 [|https-jsse-nio-443-exec-4-ReceiveLoop-254285513] ERROR c.e.e.n.c.LayeredProtocolConnection - LPC-254285513 #e1a663711aca639e [xxx.xxx.xxx.xxx:8611]: IOException in receive loop: message=null, e=java.io.EOFException, cause=null
06:30:21.635 [|https-jsse-nio-443-exec-5-ReceiveLoop-838015790] ERROR c.e.e.n.c.LayeredProtocolConnection - LPC-838015790 #ed300a36c5228121 [xxx.xxx.xxx.xxx:8611]: IOException in receive loop: message=null, e=java.io.EOFException, cause=null
06:30:23.475 [|https-jsse-nio-443-exec-11] ERROR c.e.s.s.w.c.SpecialImplController - Got an exception in handleException
org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Java heap space
  • Finalmente block-gateway-1 O pod também relata OutOfMemoryError. 
    07:03:30.663 [|https-jsse-nio-443-exec-15] WARN  o.s.w.s.m.m.a.ExceptionHandlerExceptionResolver - Failure in @ExceptionHandler com.emc.s3g.scaleio.web.controller.RestApiController#handleException(Exception, HttpServletRequest, HttpServletResponse)
java.lang.OutOfMemoryError: Java heap space
07:03:42.234 [|https-jsse-nio-443-exec-5-ReceiveLoop-793498724] ERROR c.e.e.n.c.LayeredProtocolConnection - LPC-793498724 #7f7d737257ceb2e0 [10.41.209.52:8611]: Problem with message deserialization
java.lang.OutOfMemoryError: Java heap space
07:03:42.234 [|https-jsse-nio-443-exec-12] ERROR c.e.s.s.s.ConnectionServiceImpl - Got rc MESSAGE_DESERIALIZATION_FAILED for command QueryPoll
  • catalina.out Os logs indicam falhas em HostConfig.checkResources(), sugerindo exaustão de memória durante a validação de recursos. 
    16-Jan-2025 06:31:21.628 SEVERE [Catalina-utility-2] org.apache.catalina.core.ContainerBase.threadStart Exception processing background thread
        Caused by: java.lang.OutOfMemoryError: Java heap space
                at org.apache.catalina.startup.HostConfig.checkResources(HostConfig.java:1298)
  • As medições de recursos não mostram um consumo significativo de CPU ou memória pelos pods de gateway afetados. 
    Namespace                   Name                                                     CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
---------                   ----                                                     ------------  ----------  ---------------  -------------  ---
powerflex                   block-legacy-gateway-1                                   0 (0%)        0 (0%)      0 (0%)           0 (0%)         56d
powerflex                   ciam-service-8c98989cb-5prsg                             100m (0%)     0 (0%)      128Mi (0%)       0 (0%)         56d
powerflex                   block-legacy-gateway-0                                   0 (0%)        0 (0%)      0 (0%)           0 (0%)         56d

Impacto

  • As chamadas de API para PFMP falham intermitentemente, afetando potencialmente as operações de automação e gerenciamento.
  • Os pods de gateway param de responder devido ao esgotamento de espaço de heap do Java.
  • A intervenção manual é necessária para restaurar o serviço.

 

Cause

This issue is triggered by block-legacy-gateway POST requests and the following API requests: "/api/gatewayLogin", "/api/logout", and "/api/gatewayLogout". The pod handling these requests does not fully release the associated memory upon completion, leading to a memory leak.

 

Resolution

Reinicie o afetado block-gateway Pods usando o seguinte comando:

kubectl -n powerflex rollout restart statefulset block-legacy-gateway

 

 

Additional Information

Versões afetadas

PFMP v4.6.0.x e v4.6.1

Correção feita na versão

PFMP v4.6.2 e superior

 

Article Properties
Article Number: 000297144
Article Type: Solution
Last Modified: 24 Mar 2025
Version:  1
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.