Isilon: OneFS: Panic *** FAILED ASSERTION new_num_buckets @ /b/mnt/src/sys/kern/isi_hash.c:745: Integer rollover: 0

Summary: Isilon Node Panic *** FAILED ASSERTION new_num_buckets @ /b/mnt/src/sys/kern/isi_hash.c:745: Integer rollover: 0

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.
Check out other resources

Symptoms

Node 1 rebooted by Panic 

2021-04-09T17:59:07+02:00 <0.4> AXx-2(id2) /boot/kernel.amd64/kernel: [gmp_info.c:1864](pid 31115="kt: rtxn_split")(tid=100629) group change: <2,1503> [up: 17 nodes, down: 1 node] (node 1 changed to down, node 1 service smb changed to DOWN, node 1 service nfs changed to DOWN, node 1 service all_enabled_protocols changed to DOWN)
2021-04-09T18:16:18+02:00 <0.4> AXx-2(id2) /boot/kernel.amd64/kernel: [gmp_info.c:1864](pid 34877="kt: gmp-merge")(tid=100436) group change: <1,1506> [up: 18 nodes] (node 1 changed to up)
2021-04-09T18:16:30+02:00 <0.4> AXx-2(id2) /boot/kernel.amd64/kernel: [gmp_info.c:1864](pid 34185="kt: gmp-service-upd")(tid=100425) group change: <1,1507> [up: 18 nodes] (node 1 service smb changed to UP, node 1 service nfs changed to UP)
2021-04-09T18:16:31+02:00 <0.4> AXx-2(id2) /boot/kernel.amd64/kernel: [gmp_info.c:1864](pid 35205="kt: gmp-service-upd")(tid=102668) group change: <1,1508> [up: 18 nodes] (node 1 service all_enabled_protocols changed to UP)
#Panic Stack 
panic @ time 1617983925.526, thread 0xfffff82b93b04780: Assertion Failure
cpuid = 7
Panic occurred in module kernel loaded at 0xffffffff80200000:

Stack: --------------------------------------------------
kernel:isi_assert_halt+0x2e
kernel:isi_hash_resize+0x9d
kernel:av_enqueue_work+0x162
kernel:bam_av_enqueue_nonblocking_scan+0x10a
kernel:bam_close+0x331
kernel:ifs_vnop_wrapclose+0x6d
kernel:VOP_CLOSE_APV+0xa4
kernel:vn_close+0x13f
kernel:vn_closefile+0x71
kernel:_fdrop+0x58
kernel:closef+0x296
kernel:closefp+0xad
kernel:amd64_syscall+0x396
--------------------------------------------------
*** FAILED ASSERTION new_num_buckets @ /b/mnt/src/sys/kern/isi_hash.c:745: Integer rollover: 0
Dumping stacks (40960 bytes)

Cause

It's a known issue in this OneFS version. 8.1.0.4

What is happening in general is the system isn't handling the incoming work items fast enough.  If it gets too far behind it builds up a backlog of timed out scan requests and it can't be cleared because of the incoming requests coming in faster
than the timeouts are removed.  At this point the anti-virus scanning is broken and the queues continue to build until memory is exhausted.

Resolution

The final fix has been shipped on 8.1.2 . 
Upgrade OneFS to any version above 8.1.2.0 with latest patches


Affected Products

Isilon, PowerScale OneFS
Article Properties
Article Number: 000185415
Article Type: Solution
Last Modified: 06 Sep 2022
Version:  3
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.