Unsolved
This post is more than 5 years old
9 Posts
3
5927
How to calculate the amount of files
There are numerous reasons why one would like to know how many files are being stored on an Isilon File System (IFS) or a specific sub-directory.
In many cases using standard host based tools like 'ls', 'find', or 'dir' simply take too long because of the protocol and network overhead or simply because it takes too long to walk in real time an entire tree made of millions of files. Below are a few ways tools and commands to query OneFS pre-collected statistics to find out file count:
1. Isilon 'isi statistics' command line offers a rich set of performance and capacity statistics. One less known statistics is the inode count stored on each drives in OneFS. An inode represents a single mirror of a file's logical inode (LIN). The number of LINs is almost identical to the number of files in OneFS (there are a few special systems LINs that are also counted). If we can calculate the number of inodes we can deduce the number of LINs/Files in OneFS. inode mirror count are derived from the protection level of a directory or the entire IFS. In a typical cluster with a default protection level of +2:1 the inode mirror count is 3x (3 inode copies to protect agains 2 drive failures). So we can simply devide the number of inodes by 3 and get a good approximation of the number of LINs/Files in IFS. To get the number of inodes we can simply run the following command:
se-sandbox2-1# isi statistics drive --nodes all --long |
Drive Type OpsIn BytesIn SizeIn OpsOut BytesOut SizeOut TimeAvg Slow TimeInQ Queued Busy Used Inodes
LNN:bay N/s B/s B N/s B/s B ms N/s ms % %
1:1 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.9 1.2K
1:2 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.3K
1:3 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.2K
1:4 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.4K
1:5 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.8 1.3K
1:6 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.4K
1:7 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.3K
1:8 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.5K
1:9 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.1K
1:10 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.3K
1:11 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.3K
1:12 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.6K
2:1 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.9 1.4K
2:2 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.4K
2:3 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.6K
2:4 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.5K
2:5 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.8 1.4K
2:6 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.4K
2:7 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.3K
2:8 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.5K
2:9 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.5K
2:10 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.4K
2:11 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.3K
2:12 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.3K
3:1 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.9 1.4K
3:2 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.2K
3:3 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.2K
3:4 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.4K
3:5 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.4K
3:6 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 1.9 1.3K
3:7 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.3K
3:8 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.5K
3:9 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.3K
3:10 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.7K
3:11 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.5K
3:12 SATA 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 2.9 1.3K
In the output above the right most value is the inode count. With an average inodecount of 1.4K files we get: 1.4K x 36 = 50.4K inodes. We divide it by 3 and get 16,800 LINs/Files in this example under IFS.
Note: with SSDs the default layout is to have a single copy on SSD and the rest on the HDDs. So in this case we can simply count the number of inodes on the SSD drives which equals the number of LINs/Files.
For example:
se-sandbox1-1# isi statistics drive --nodes all --long --type ssd
Drive Type OpsIn BytesIn SizeIn OpsOut BytesOut SizeOut TimeAvg Slow TimeInQ Queued Busy Used Inodes
LNN:bay N/s B/s B N/s B/s B ms N/s ms % %
1:1 SSD 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.9 1.6K
2:1 SSD 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.9 1.6K
3:1 SSD 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.9 1.6K
4:1 SSD 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.9 1.6K
5:1 SSD 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.9 1.6K
6:1 SSD 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.9 1.6K
7:1 SSD 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.9 1.6K
8:1 SSD 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.9 1.6K
In this example we have 12.8K inodes on SSDs which equals the number of LINs/Files in IFS (single inode mirror on SSD per file) this is directly translates to 12,800 files.
There are other methods that can use licensed features of OneFS like the file count from SmartQuotas or File Count from InsightIQ File Systems Properties reports.
Shai Harmelin
NAS Specialists
Peter_Sero
1.2K Posts
2
September 26th, 2013 01:00
Great explanation behind the scenes!
Curious, did you notice how unbalanced the disk usage is,
both for Used% and for #Inodes, with both numbers not even being correlated?
There's a similar situation on our cluster, even after Multiscan/AutoBalance
(Used%: good balance as expected, but #Inodes from 1.3M to 3.0M within same pool and disk type).
I was preparing to contact support. Any clues from your side?
BTW, one more, and built-in, method for counting the LINs is:
isi job start LinCount
(takes only a couple of minutes on high density nodes)
isi job hist -v
-- Peter