ECS: xDoctor: RAP178 - java.lang.NullPointerException in JournalChunkGCTaskScanner.java

Résumé: Garbage Collection fails to reclaim Journal chunks due to java.lang.NullPointerException.

Cet article concerne Cet article ne concerne pas Cet article n’est associé à aucun produit spécifique. Toutes les versions du produit ne sont pas identifiées dans cet article.
Consulter d’autres ressources

Symptômes

  • High L2 metadata utilization
  • High journal utilization
  • High capacity L2 utilization

Chunk manager (cm.log) logs the following error: java.lang.NullPointerException

2024-01-16T16:38:57,443 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-117] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
        at com.emc.storageos.data.object.devicescanner.RepGroupObserverImpl.getFailedZones(RepGroupObserverImpl.java:781)
        at com.emc.storageos.data.chunkmanager.JournalChunkGCTaskScanner.isFailedZone(JournalChunkGCTaskScanner.java:360)
        at com.emc.storageos.data.chunkmanager.JournalChunkGCTaskScanner.runTask(JournalChunkGCTaskScanner.java:237)
        at com.emc.storageos.data.object.task.TaskScanner.scan(TaskScanner.java:81)
        at com.emc.storageos.data.object.Scanner$ScannerThread.run(Scanner.java:269)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:131)
        at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:74)
        at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:82)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

Verify that the Journal Chunk Task Scanner is throwing an exception in the cm.log files:
# svc_log -f "JournalChunkGCTaskScanner.java" "NullPointerException" -files cm-error.log* -start 10m

admin@ecs-node:~> svc_log -f "JournalChunkGCTaskScanner.java" "Exception" -files cm-error.log* -start 10m
svc_log v1.0.33 (svc_tools v2.20.0)                 Started 2024-12-13 11:51:29

Running on nodes:                        <All nodes>
Time range:                              2024-12-13 11:41:29 - 2024-12-13 11:51:29
Filter string(s):                        'JournalChunkGCTaskScanner.java' AND 'Exception'
File(s) to search:                               cm-error.log*

Show nodename(s):                        True
Show nodename(s):                        True

169.254.1.1 2024-12-13T11:43:54,021 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-074] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.1 2024-12-13T11:43:54,022 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-074] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.1 2024-12-13T11:43:54,036 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-074] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
.......
169.254.1.2 2024-12-13T11:50:18,135 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-282] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.2 2024-12-13T11:50:18,144 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-282] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.2 2024-12-13T11:50:18,149 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-282] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
.......
169.254.1.6 2024-12-13T11:49:43,389 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-080] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.6 2024-12-13T11:49:43,390 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-080] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
169.254.1.6 2024-12-13T11:49:43,411 [TaskScheduler-ChunkManager-DEFAULT_BACKGROUND_OPERATION-ScheduledExecutor-080] ERROR  JournalChunkGCTaskScanner.java (line 338) Exception from garbageCollect
java.lang.NullPointerException
.......

# svc_log -f "JournalChunkGCTaskScanner.java" "NullPointerException" -files 'cm-error.log' -start 1d -st hour -sh*

admin@ecs-node:~> svc_log -f "JournalChunkGCTaskScanner.java" "NullPointerException" -files 'cm-error.log' -start 1d -st hour -sh*
svc_log v1.0.33 (svc_tools v2.20.0)                 Started 2024-12-13 11:45:46

Running on nodes:                        <All nodes>
Time range:                              2024-12-12 11:45:46 - 2024-12-13 11:45:46
Filter string(s):                        'JournalChunkGCTaskScanner.java' AND 'NullPointerException'
File(s) to search:                               cm-error.log*

Show nodename(s):                        True
Show nodename(s):                        True

Count of message occurrences per hour:

                    # Log
Time                Entries
---------------------------
2024-12-12 11:xx    58500
2024-12-12 12:xx    224100
2024-12-12 13:xx    224203
2024-12-12 14:xx    224296
2024-12-12 15:xx    223711
2024-12-12 16:xx    224017
2024-12-12 17:xx    224071
2024-12-12 18:xx    224322
2024-12-12 19:xx    225198
2024-12-12 20:xx    224965
2024-12-12 21:xx    224214
2024-12-12 22:xx    224906
2024-12-12 23:xx    225339
2024-12-13 00:xx    224910
2024-12-13 01:xx    225692
2024-12-13 02:xx    225735
2024-12-13 03:xx    225804
2024-12-13 04:xx    226018
2024-12-13 05:xx    226271
2024-12-13 06:xx    226475
2024-12-13 07:xx    225937
2024-12-13 08:xx    226782
2024-12-13 09:xx    226879
2024-12-13 10:xx    226409
2024-12-13 11:xx    158538

Total:              5397292

Cause

Null Pointer Exception errors in JournalChunkGCTaskScanner can cause Journal Garbage Collection issue ultimately leading to high metadata over head.

Résolution

Upgrade ECS Appliance software to 3.8.0.6 or later.

Produits concernés

ECS, ECS Appliance, ECS Appliance Gen 1, ECS Appliance Gen 2, ECS Appliance Gen 3

Produits

ECS Appliance Hardware Gen3 EX5000, ECS Appliance Hardware Gen3 EX300, ECS Appliance Hardware Gen3 EX3000, ECS Appliance Hardware Gen1 U-Series, ECS Appliance Hardware Gen1 C-Series, ECS Appliance Hardware Gen2 C-Series , ECS Appliance Hardware Gen2 D-Series, ECS Appliance Hardware Gen2 U-Series, ECS Appliance Hardware Gen3 EX500, ECS Appliance Hardware Gen3 EXF900, ECS Appliance Hardware Series, ECS Appliance Software with Encryption, ECS Appliance Software without Encryption, ECS Software ...
Propriétés de l’article
Numéro d’article: 000227107
Type d’article: Solution
Dernière modification: 10 sept. 2025
Version:  4
Trouvez des réponses à vos questions auprès d’autres utilisateurs Dell
Services de support
Vérifiez si votre appareil est couvert par les services de support.