This post is more than 5 years old
115 Posts
0
3494
Dir/File Protection
hello,
i know there are lot of posts in community around this but its great if someone clarify this protection below based on size.
Number of Nodes: 7
protection level +2d:1n
folder name: ztest
file name with in folder : file1
folder < 128KB so it should just mirror across nodes so it should be 7x correct.
file1 size is 176B so not sure why the number of stripes is grouped in a set of 2 ( /2 number ).
increase the dir protection to higher requested protection than its contect is checked/set on cluster
isilon-prod# isi get /ifs | grep -i ztest
default 4x concurrency on ztest/
isilon-prod# ls -lah /ifs/ztest
total 96
drwxr-xr-x 2 root wheel 23B Jul 29 13:45 .
drwxrwxrwx 60 root wheel 1.7K Jul 29 13:55 ..
-rw-r--r-- 1 root wheel 176B Jul 29 13:55 file1
isilon-prod# isi get /ifs/ztest/
POLICY LEVEL PERFORMANCE COAL FILE
default 4x concurrency on ./
default 8x/2 concurrency on ../
default 6+2/2 concurrency on file1
kipcranford
125 Posts
0
July 29th, 2016 14:00
> folder < 128KB so it should just mirror across nodes so it should be 7x correct.
Not quite. Directories (and file metadata) are always mirrored, regardless of whether a FEC scheme is in use. So in your example, it's mirrored by default, not because it's < 128 KiB.
Also, directories are mirrored at one level higher than the protection that's in place. So in your example of +2d:1n, that directory will have 4 mirrors.
> file1 size is 176B so not sure why the number of stripes is grouped in a set of 2 ( /2 number )
A file that small will be *effectively* mirrored because the FEC algorithm doesn't have enough file data to actually create stripes. So this file, in your example, will have 3 mirrors, each 176 B in size even though the protection policy on the file is +2d:1n.
> default 6+2/2 concurrency on file1
And this is the example of what I said above. You have a 4-node cluster and are using the default FEC protection policy of +2d:1n. This means that OneFS can create stripes of 6+2/2, if it has the data to do so. Your "file1" at 176 B doesn't have the necessary data, so OneFS will 3x mirror it (but doesn't change the *protection policy*). If you were to append to the file to grow it beyond 128 KiB, OneFS would then switch from creating mirrors to creating FEC stripes.
kipcranford
125 Posts
0
July 29th, 2016 15:00
> ours is 7 Node cluster not 4 but ( Node 1-4 ---> X200 && Node 5-7 ---> NL400 ) we have 2 node pools.
The pool you're writing to (in your example) is 4 nodes -- I can tell by the 6+2/2 protection. The fact that you have other pools in the cluster isn't germane to this particular discussion about protection.
> but its not mirroring though rather i see its striping ( default 6+2/2 concurrency on file1 )
Well, yes and no, but that was point about the "effective protection" in my first comment. The protection policy, i.e. what you're telling OneFS you want the file protected at, is FEC (+2d:1n). The best protection that OneFS can actually do, given the size of the file, is a 3x mirror. OneFS will not start creating stripes until files are > 128 KiB in size.
To prove that you have a 3x mirrored file, just do a "du -sh" on the file. You should see 3 * 8 KiB + some metadata (like inodes), or around 26 KiB in total.
Raj_la
115 Posts
0
July 29th, 2016 15:00
ours is 7 Node cluster not 4 but ( Node 1-4 ---> X200 && Node 5-7 ---> NL400 ) we have 2 node pools.
>Your "file1" at 176 B doesn't have the necessary data, so OneFS will 3x mirror it (but doesn't change the *protection policy*).
but its not mirroring though rather i see its striping ( default 6+2/2 concurrency on file1 )
Raj_la
115 Posts
0
August 3rd, 2016 15:00
thank you for the feedback. it would be grateful if you please explain this scenario for me. here i attach the screenshot ( snippet from overhead pic ) if we talk about example of second row/column
4 Nodes with +2:1 protection how we can 6+2 equation derived , in this 6+2 i understand +2 but how we get the number 6.
also 3rd row of 5 Nodes how we get number 8.
appreciate your help!
Peter_Sero
1.2K Posts
1
August 4th, 2016 02:00
4+2, 6+2, 8+2 and so on represent the numbers of disks in a stripe group.
4+2/2 is the OneFS "idiom" for (4+2)/2, which mathematically is 3, just the number of nodes in this example.
All the 2s come from the number of drives allowed to fail.
Using variables:
d := number of drives allowed to fail (= 2 in the examples)
N := total number of nodes in pool (3, 4, 5, ...)
n := number of nodes allowed to fail (= 1 in the examples)
X := d * (N - n) the number you asked for
things can be expressed in more general form:
4+2/2 becomes: (X + d*n) / d = N <-- always the number of nodes
4+2 becomes: X + d*n = N * d <-- always the number of disks in a stripe group
Cheers
-- Peter
Raj_la
115 Posts
0
August 4th, 2016 13:00
Nice got it - thank you so much ford & Peter.
kipcranford
125 Posts
0
August 9th, 2016 15:00
>
> 4 Nodes with +2:1 protection how we can 6+2 equation derived , in this 6+2 i understand +2 but how we get the number 6.
> also 3rd row of 5 Nodes how we get number 8.
>
I just added a blog post (with illustrations!) that shows what OneFS does for layout in the event of small files and/or over-protected clusters (i.e. small cluster, high protection). Hopefully the blog format allows for more detail than what can be conveyed here. The post is at:
https://community.emc.com/blogs/kip_cranford/2016/08/09/onefs-protection-policy-vs-actual-protection-level
idioms
1 Message
0
January 19th, 2023 21:00
It was very helpful, thanks!