ECS: CAS-fout -10204 - Hard quotum bucket overschreden
Summary: CAS-fout (Content Addressable Storage): -10204 wordt weergegeven in het KPI-script omdat het harde quotum van de CAS-bucket is overschreden.
Symptoms
Het Filenet-applicatieteam heeft een probleem gemeld bij het schrijven van blobs naar het ECS.
Het Filenet-logboekbestand is opgevraagd vanaf het moment van het incident en de onderstaande fout (-10204) is gekoppeld aan "the use of this operation is restricted" werden gezien in de Filenet-logboeken:
[6/29/20 13:36:50:025 BST] 00000117 SystemOut O 2020-06-29 13:36:50,025 ERROR [ContentQueueWorker_PERSONALLN_S_Worker background task processing282] - Failed to create new tag FN.3029D828-A10B-4719-BEA1-6BA4357E4FD3.0 (Centera error code: -10204) com.filepool.fplibrary.FPLibraryException: The use of this operation is restricted (transid='sl008509/644175/WRITE_BLOB') at com.filepool.fplibrary.FPTag.BlobWrite(Unknown Source) at com.filenet.engine.content.fcprovider.centera.CenteraAdaptor.addTagAndContent(CenteraAdaptor.java:2188) at com.filenet.engine.content.fcprovider.centera.CenteraAdaptor.saveDocument(CenteraAdaptor.java:333) at com.filenet.engine.content.fcprovider.centera.CenteraFCP.createMultipart2(CenteraFCP.java:367) at com.filenet.engine.content.FixedContentArea.rollfwdMigrateContent(FixedContentArea.java:1865) at com.filenet.engine.content.FixedContentArea.executeRollFwdCommand(FixedContentArea.java:1617) at com.filenet.engine.content.BaseContentArea.executeRollFwdCommand(BaseContentArea.java:1805) at com.filenet.engine.queueitem.ContentQueueWorker.processBatch(ContentQueueWorker.java:463) at com.filenet.engine.queueitem.ContentQueueWorker.execute(ContentQueueWorker.java:160) at com.filenet.engine.tasks.BackgroundTask.safeExecute(BackgroundTask.java:258) at com.filenet.engine.tasks.BackgroundTask$BackgroundTaskPriviledgedExceptionAction.run(BackgroundTask.java:1090) at com.filenet.engine.context.CallState.doAsSystem(CallState.java:549) at com.filenet.engine.tasks.BackgroundTask.run(BackgroundTask.java:210) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:949) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:973) at java.lang.Thread.run(Thread.java:790) [6/29/20 13:36:50:026 BST] 00000117 SystemOut O 18734167 [ContentQueueWorker_PERSONALLN_S_Worker background task processing282] ERROR filenet_error.fixed_content_provider.com.filenet.engine.content.fcprovider.centera.CenteraAdaptor - Failed to create new tag FN.3029D828-A10B-4719-BEA1-6BA4357E4FD3.0 (Centera error code: -10204) com.filepool.fplibrary.FPLibraryException: The use of this operation is restricted (transid='sl008509/644175/WRITE_BLOB') at com.filepool.fplibrary.FPTag.BlobWrite(Unknown Source) at com.filenet.engine.content.fcprovider.centera.CenteraAdaptor.addTagAndContent(CenteraAdaptor.java:2188) at com.filenet.engine.content.fcprovider.centera.CenteraAdaptor.saveDocument(CenteraAdaptor.java:333) at com.filenet.engine.content.fcprovider.centera.CenteraFCP.createMultipart2(CenteraFCP.java:367) at com.filenet.engine.content.FixedContentArea.rollfwdMigrateContent(FixedContentArea.java:1865) at com.filenet.engine.content.FixedContentArea.executeRollFwdCommand(FixedContentArea.java:1617) at com.filenet.engine.content.BaseContentArea.executeRollFwdCommand(BaseContentArea.java:1805) at com.filenet.engine.queueitem.ContentQueueWorker.processBatch(ContentQueueWorker.java:463) at com.filenet.engine.queueitem.ContentQueueWorker.execute(ContentQueueWorker.java:160) at com.filenet.engine.tasks.BackgroundTask.safeExecute(BackgroundTask.java:258) at com.filenet.engine.tasks.BackgroundTask$BackgroundTaskPriviledgedExceptionAction.run(BackgroundTask.java:1090) at com.filenet.engine.context.CallState.doAsSystem(CallState.java:549) at com.filenet.engine.tasks.BackgroundTask.run(BackgroundTask.java:210) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:949) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:973) at java.lang.Thread.run(Thread.java:790)
De KPI-opdracht uitgevoerd om te controleren op de huidige status van CAS-bewerkingen, en ontdekte dat 73,53% van het totale aantal schrijf-blobbewerkingen mislukte met foutcode -10204:
# kpi.sh -cas -s -errorcode "-10204" -start "2 days ago"
kpi.sh Version 2.5.14
Report start: 2020-06-27 14:19:10 Report end: 2020-06-29 14:19:10
Extracting RequestLog data: DONE
Preprocessing data: DONE
All Requests -10204 Errors
READ READ WRITE WRITE READ READ WRITE WRITE
Node CLIP BLOB CLIP BLOB DELETE Total CLIP BLOB CLIP BLOB DELETE Total
10.xxx.xxx.xxx 325587 413917 124736 435750 16198 1316188 0 0 0 321319 301 321620
10.xxx.xxx.xxx 334412 405561 130012 440337 9245 1319567 0 0 0 322607 301 322908
10.xxx.xxx.xxx 323114 414638 133056 437784 9348 1317940 0 0 0 321415 308 321723
10.xxx.xxx.xxx 335851 407194 120876 435475 17422 1316818 0 0 0 320809 304 321113
10.xxx.xxx.xxx 327851 409858 128350 431039 16141 1313239 0 0 0 316541 310 316851
10.xxx.xxx.xxx 320987 410635 136813 441088 8477 1318000 0 0 0 323263 301 323564
10.xxx.xxx.xxx 319899 409053 138668 434242 8257 1310119 0 0 0 319452 307 319759
10.xxx.xxx.xxx 337687 409034 119097 435702 17143 1318663 0 0 0 321973 308 322281
Req Totals: 2625388 3279890 1031608 3491417 102231 10530534 0 0 0 2567379 2440 2569819
Req Error %: 0.00 0.00 0.00 73.53 2.39 24.40Cause
Het probleem wordt veroorzaakt doordat het harde quotum van de bucket is overschreden, waardoor de schrijfbewerkingen niet zijn toegestaan.
Resolution
Het harde quotum van de bucket moet worden uitgebreid om de schrijfbewerkingen te hervatten.
KB ECS: Bucket instellen Zachte en harde quota en ECS: Hoe de quotaset voor een ECS-bucket kan worden verhoogd, moet worden gevolgd.