PowerFlex 管理平台：PowerFlex 閘道 API 呼叫因 Java 堆積空間而失敗

Summary: 由於 PowerFlex Gateway Pod 中的 Java 堆積空間耗盡，OpenStack 的 API 呼叫以及與 PowerFlex 管理平台 （PFMP） 的其他類似整合失敗。重新啟動受影響的區塊閘道 Pod 可暫時解決問題。

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

  • PowerFlex Manager （PFMP） 的 API 呼叫間歇性失敗，影響 OpenStack 和其他整合。
  • block-gateway-0 容器報告 OutOfMemoryError，請求將重定向到 block-gateway-1。 
    06:30:21.635 [|https-jsse-nio-443-exec-4-ReceiveLoop-254285513] ERROR c.e.e.n.c.LayeredProtocolConnection - LPC-254285513 #e1a663711aca639e [xxx.xxx.xxx.xxx:8611]: IOException in receive loop: message=null, e=java.io.EOFException, cause=null
06:30:21.635 [|https-jsse-nio-443-exec-5-ReceiveLoop-838015790] ERROR c.e.e.n.c.LayeredProtocolConnection - LPC-838015790 #ed300a36c5228121 [xxx.xxx.xxx.xxx:8611]: IOException in receive loop: message=null, e=java.io.EOFException, cause=null
06:30:23.475 [|https-jsse-nio-443-exec-11] ERROR c.e.s.s.w.c.SpecialImplController - Got an exception in handleException
org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.OutOfMemoryError: Java heap space
  • 最終， block-gateway-1 Pod 還會報告 OutOfMemoryError。 
    07:03:30.663 [|https-jsse-nio-443-exec-15] WARN  o.s.w.s.m.m.a.ExceptionHandlerExceptionResolver - Failure in @ExceptionHandler com.emc.s3g.scaleio.web.controller.RestApiController#handleException(Exception, HttpServletRequest, HttpServletResponse)
java.lang.OutOfMemoryError: Java heap space
07:03:42.234 [|https-jsse-nio-443-exec-5-ReceiveLoop-793498724] ERROR c.e.e.n.c.LayeredProtocolConnection - LPC-793498724 #7f7d737257ceb2e0 [10.41.209.52:8611]: Problem with message deserialization
java.lang.OutOfMemoryError: Java heap space
07:03:42.234 [|https-jsse-nio-443-exec-12] ERROR c.e.s.s.s.ConnectionServiceImpl - Got rc MESSAGE_DESERIALIZATION_FAILED for command QueryPoll
  • catalina.out 記錄顯示故障 HostConfig.checkResources()，表示在資源驗證期間記憶體耗盡。 
    16-Jan-2025 06:31:21.628 SEVERE [Catalina-utility-2] org.apache.catalina.core.ContainerBase.threadStart Exception processing background thread
        Caused by: java.lang.OutOfMemoryError: Java heap space
                at org.apache.catalina.startup.HostConfig.checkResources(HostConfig.java:1298)
  • 資源指標顯示受影響的閘道 Pod 不會消耗大量 CPU 或記憶體。 
    Namespace                   Name                                                     CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
---------                   ----                                                     ------------  ----------  ---------------  -------------  ---
powerflex                   block-legacy-gateway-1                                   0 (0%)        0 (0%)      0 (0%)           0 (0%)         56d
powerflex                   ciam-service-8c98989cb-5prsg                             100m (0%)     0 (0%)      128Mi (0%)       0 (0%)         56d
powerflex                   block-legacy-gateway-0                                   0 (0%)        0 (0%)      0 (0%)           0 (0%)         56d

影響

  • API 呼叫 PFMP 會間歇性失敗，可能會影響自動化和管理作業。
  • 由於 Java 堆積空間耗盡，閘道 Pod 沒有回應。
  • 需要手動干預才能恢復服務。

 

Cause

This issue is triggered by block-legacy-gateway POST requests and the following API requests: "/api/gatewayLogin", "/api/logout", and "/api/gatewayLogout". The pod handling these requests does not fully release the associated memory upon completion, leading to a memory leak.

 

Resolution

重新啟動受影響的 block-gateway 使用以下命令的 Pod：

kubectl -n powerflex rollout restart statefulset block-legacy-gateway

 

 

Additional Information

受影響的版本

PFMP v4.6.0.x 和 v4.6.1

已修正問題的版本

PFMP v4.6.2 及更新版本

 

Article Properties
Article Number: 000297144
Article Type: Solution
Last Modified: 24 Mar 2025
Version:  1
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.