ECS: xDoctor: RAP178 - java.lang.NullPointerException in JournalChunkGCTaskScanner.java

摘要: Garbage Collection fails to reclaim Journal chunks due to java.lang.NullPointerException.

本文章適用於 本文章不適用於 本文無關於任何特定產品。 本文未識別所有產品版本。
查看其他資源

症狀

  • High L2 metadata utilization
  • High journal utilization
  • High capacity L2 utilization

Chunk manager (cm.log) logs the following error: java.lang.NullPointerException

2024-01-16T16:38:57,443 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-117] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
        at com.emc.storageos.data.object.devicescanner.RepGroupObserverImpl.getFailedZones(RepGroupObserverImpl.java:781)
        at com.emc.storageos.data.chunkmanager.JournalChunkGCTaskScanner.isFailedZone(JournalChunkGCTaskScanner.java:360)
        at com.emc.storageos.data.chunkmanager.JournalChunkGCTaskScanner.runTask(JournalChunkGCTaskScanner.java:237)
        at com.emc.storageos.data.object.task.TaskScanner.scan(TaskScanner.java:81)
        at com.emc.storageos.data.object.Scanner$ScannerThread.run(Scanner.java:269)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

Verify that the Journal Chunk Task Scanner is throwing an exception in the cm.log files:
# svc_log -f "JournalChunkGCTaskScanner.java" "NullPointerException" -files cm-error.log* -start 10m

admin@ecs-node:~> svc_log -f "JournalChunkGCTaskScanner.java" "Exception" -files cm-error.log* -start 10m
svc_log v1.0.33 (svc_tools v2.20.0)                 Started 2024-12-13 11:51:29

Running on nodes:                        <All nodes>
Time range:                              2024-12-13 11:41:29 - 2024-12-13 11:51:29
Filter string(s):                        'JournalChunkGCTaskScanner.java' AND 'Exception'
File(s) to search:                               cm-error.log*

Show nodename(s):                        True
Show nodename(s):                        True

169.254.1.1 2024-12-13T11:43:54,021 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-074] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.1 2024-12-13T11:43:54,022 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-074] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.1 2024-12-13T11:43:54,036 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-074] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
.......
169.254.1.2 2024-12-13T11:50:18,135 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-282] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.2 2024-12-13T11:50:18,144 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-282] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.2 2024-12-13T11:50:18,149 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-282] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
.......
169.254.1.6 2024-12-13T11:49:43,389 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-080] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.6 2024-12-13T11:49:43,390 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-080] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.6 2024-12-13T11:49:43,411 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-080] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
.......

# svc_log -f "JournalChunkGCTaskScanner.java" "NullPointerException" -files 'cm-error.log' -start 1d -st hour -sh*

admin@ecs-node:~> svc_log -f "JournalChunkGCTaskScanner.java" "NullPointerException" -files 'cm-error.log' -start 1d -st hour -sh*
svc_log v1.0.33 (svc_tools v2.20.0)                 Started 2024-12-13 11:45:46

Running on nodes:                        <All nodes>
Time range:                              2024-12-12 11:45:46 - 2024-12-13 11:45:46
Filter string(s):                        'JournalChunkGCTaskScanner.java' AND 'NullPointerException'
File(s) to search:                               cm-error.log*

Show nodename(s):                        True
Show nodename(s):                        True

Count of message occurrences per hour:

                    # Log
Time                Entries
---------------------------
2024-12-12 11:xx    58500
2024-12-12 12:xx    224100
2024-12-12 13:xx    224203
2024-12-12 14:xx    224296
2024-12-12 15:xx    223711
2024-12-12 16:xx    224017
2024-12-12 17:xx    224071
2024-12-12 18:xx    224322
2024-12-12 19:xx    225198
2024-12-12 20:xx    224965
2024-12-12 21:xx    224214
2024-12-12 22:xx    224906
2024-12-12 23:xx    225339
2024-12-13 00:xx    224910
2024-12-13 01:xx    225692
2024-12-13 02:xx    225735
2024-12-13 03:xx    225804
2024-12-13 04:xx    226018
2024-12-13 05:xx    226271
2024-12-13 06:xx    226475
2024-12-13 07:xx    225937
2024-12-13 08:xx    226782
2024-12-13 09:xx    226879
2024-12-13 10:xx    226409
2024-12-13 11:xx    158538

Total:              5397292

原因

Null Pointer Exception errors in JournalChunkGCTaskScanner can cause Journal Garbage Collection issue ultimately leading to high metadata over head.

解析度

Upgrade ECS Appliance software to 3.8.0.6 or later.

受影響的產品

ECS, ECS Appliance, ECS Appliance Gen 1, ECS Appliance Gen 2, ECS Appliance Gen 3

產品

ECS Appliance Hardware Gen3 EX5000, ECS Appliance Hardware Gen3 EX300, ECS Appliance Hardware Gen3 EX3000, ECS Appliance Hardware Gen1 U-Series, ECS Appliance Hardware Gen1 C-Series, ECS Appliance Hardware Gen2 C-Series , ECS Appliance Hardware Gen2 D-Series, ECS Appliance Hardware Gen2 U-Series, ECS Appliance Hardware Gen3 EX500, ECS Appliance Hardware Gen3 EXF900, ECS Appliance Hardware Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, ECS Software ...
文章屬性
文章編號: 000227107
文章類型: Solution
上次修改時間: 10 9月 2025
版本:  4
向其他 Dell 使用者尋求您問題的答案
支援服務
檢查您的裝置是否在支援服務的涵蓋範圍內。