PPDM : Resolving Partial Indexing with Exception "The asset is partially indexed. Failed index run for disk xxxxxx"

PPDM NAS indexing job Completed with Exception "The asset is partially indexed. Failed index run for disk xxxxxx".

From the /opt/emc/vmdirect/runtime/logs/searchLogBundles/SearchNode-searchnode1.dell.com-20241016T134915077/opt/emc/search/logs/indexing-worker/indexing-worker-searchnode1.dell.com.log log, we could see the indexing job failed when sorting the metadata.

2024-10-16 11:43:03.253+0800    30      INFO    searchnode1.dell.com   4901    nas-sqlitedb.go(403)    main.convertNASSQLToFilelist    Sorting the NAS metadata filelist file
2024-10-16 11:43:03.27+0800     31      INFO    searchnode1.dell.com   4901    timeutils.go(31)        dellemc.com/search/common/ioutils.TimeTrack     SortFile time taken: 16.465611ms
2024-10-16 11:43:03.272+0800    32      INFO    searchnode1.dell.com   4901    timeutils.go(31)        dellemc.com/search/common/ioutils.TimeTrack     convertNASSQLToFilelist time taken: 3m14.624384496s
2024-10-16 11:43:03.272+0800    33      ERROR   searchnode1.dell.com   4901    assetindex.go(284)      main.assetIndex Failed Index Run[13]: Error processing SQLiteDB: ERROR:Failed to Sort file[[-T /var/log/dellemc/search/tmp /var/log/dellemc/search/tmp/CurrentMetadata4258804373.csv -o /var/log/dellemc/search/tmp/CurrentMetadata4258804373.csv]]: exit status 2
2024-10-16 11:43:03.276+0800    34      ERROR   searchnode1.dell.com   4901    indexing-worker.go(442) main.executeRun Failed to run indexing. Failed index run for disk 1729008810,
2024-10-16 11:43:03.277+0800    35      ERROR   searchnode1.dell.com   4901    jobtasks.go(342)        dellemc.com/search/common/jobtasks.(*TaskController).EndTask    Task Failed:[31] : {TZzwHzvp_V1zIQaM15CZ d9sXJC5q_V1i6SaM15AS Ihug-q3q_V1x8LaM15LU5 2024-10-16 03:12:42.185419479 +0000 UTC 2024-10-16 03:39:48.609908706 +0000 UTC 2024-10-16 03:43:03.276392124 +0000 UTC 1 6 3 searchnode1.dell.com false 0 31 Failed index run for disk 1729008810,  map[] map[assetId:1dc877dc-fd5e-5484-a5be-cc3bde6dfe4b backupId:R00oLSOMXJppA3HbB3i42ycp4C7wmq3-DK7s1nD1PrnRt4= plcId:9770751e-a086-4ae6-a4c2-d94d8aeddef8 plcJobId:93b28563-6759-4103-bfe0-e7262242f7d5] nbJQk5IBZA3oQJlyPBPJ}
2024-10-16 11:43:03.284+0800    36      ERROR   searchnode1.dell.com   4901    indexing-worker.go(753) main.endIndexingTask    Indexing Task Execution Failed : {
        "jobId": "TZzwHzvp_V1zIQaM15CZ",
        "taskId": "d9sXJC5q_V1i6SaM15AS",
        "parentId": "Ihug-q3q_V1x8LaM15LU5",
        "createTime": "2024-10-16T03:12:42.185419479Z",
        "startTime": "2024-10-16T03:39:48.609908706Z",
        "endTime": "2024-10-16T03:43:03.276392124Z",
        "jobType": 1,
        "taskType": 6,
        "state": 3,
        "executingNode": "searchnode1.dell.com",
        "retryable": false,
        "retryCount": 0,
        "returnCode": 31,
        "errorMsg": "Failed index run for disk 1729008810, ",
        "stats": {
                "itemsIndexPerSecond": 0,
                "msecsAdding": 0,

This is because during sorting, the temp directory is going out of space. 

Bug PPDM-302546 is filed to look for alternative location for temp space when the SORT of the metadata fails.

This will be permanently fixed in PPDM 19.18. 

If customer requests, engineer can provide a binary on PPDM 19.17 for the fix. Please contact Dell Support for further assistance on resolving the issue.