Isilon

[Raj_la]

hello,

i know there are lot of posts in community around this but its great if someone clarify this protection below based on size.

Number of Nodes:  7

protection level +2d:1n

folder name: ztest

file name with in  folder : file1

folder < 128KB so it should just mirror across nodes so it should be 7x correct.

file1 size is 176B so not sure why the number of stripes is grouped in a set of 2 (  /2 number ).

increase the dir protection to higher requested protection than its contect is checked/set on cluster

isilon-prod# isi get /ifs | grep -i ztest

default      4x concurrency on    ztest/

isilon-prod# ls -lah /ifs/ztest

total 96

drwxr-xr-x     2 root  wheel    23B Jul 29 13:45 .

drwxrwxrwx    60 root  wheel   1.7K Jul 29 13:55 ..

-rw-r--r--     1 root  wheel   176B Jul 29 13:55 file1

isilon-prod# isi get /ifs/ztest/

POLICY    LEVEL PERFORMANCE COAL  FILE

default      4x concurrency on    ./

default    8x/2 concurrency on    ../

default   6+2/2 concurrency on    file1

[kipcranford · Accepted solution]

> folder < 128KB so it should just mirror across nodes so it should be 7x correct.

Not quite.  Directories (and file metadata) are always mirrored, regardless of whether a FEC scheme is in use.  So in your example, it's mirrored by default, not because it's < 128 KiB.

Also, directories are mirrored at one level higher than the protection that's in place.  So in your example of +2d:1n, that directory will have 4 mirrors.

> file1 size is 176B so not sure why the number of stripes is grouped in a set of 2 (  /2 number )

A file that small will be *effectively* mirrored because the FEC algorithm doesn't have enough file data to actually create stripes.  So this file, in your example, will have 3 mirrors, each 176 B in size even though the protection policy on the file is +2d:1n.

> default   6+2/2 concurrency on    file1

And this is the example of what I said above.  You have a 4-node cluster and are using the default FEC protection policy of +2d:1n.  This means that OneFS can create stripes of 6+2/2, if it has the data to do so.  Your "file1" at 176 B doesn't have the necessary data, so OneFS will 3x mirror it (but doesn't change the *protection policy*).  If you were to append to the file to grow it beyond 128 KiB, OneFS would then switch from creating mirrors to creating FEC stripes.

[Raj_la]

ours is 7 Node cluster not 4 but  (  Node 1-4  --->  X200  && Node 5-7  ---> NL400 ) we have 2 node pools.

>Your "file1" at 176 B doesn't have the necessary data, so OneFS will 3x mirror it (but doesn't change the *protection policy*).

but its not mirroring though  rather i see its striping (  default   6+2/2 concurrency on    file1 )

[kipcranford]

> ours is 7 Node cluster not 4 but  (  Node 1-4  --->  X200  && Node 5-7  ---> NL400 ) we have 2 node pools.

The pool you're writing to (in your example) is 4 nodes -- I can tell by the 6+2/2 protection.  The fact that you have other pools in the cluster isn't germane to this particular discussion about protection.

> but its not mirroring though  rather i see its striping (  default   6+2/2 concurrency on    file1 )

Well, yes and no, but that was point about the "effective protection" in my first comment.  The protection policy, i.e. what you're telling OneFS you want the file protected at, is FEC (+2d:1n).  The best protection that OneFS can actually do, given the size of the file, is a 3x mirror.  OneFS will not start creating stripes until files are > 128 KiB in size.

To prove that you have a 3x mirrored file, just do a "du -sh" on the file.  You should see 3 * 8 KiB + some metadata (like inodes), or around 26 KiB in total.

[Raj_la]

thank you for the feedback. it would be grateful if you please explain this scenario for me. here i attach the screenshot ( snippet from overhead pic )  if we talk about  example of second row/column

4 Nodes with +2:1 protection how we can 6+2 equation derived , in this 6+2 i understand +2 but how we get the number 6.

also 3rd row of 5 Nodes how we get number 8.

appreciate your help! 

[Peter_Sero]

4+2, 6+2, 8+2  and so on represent the numbers of disks in a stripe group.

4+2/2 is the OneFS "idiom" for (4+2)/2, which mathematically is 3, just the number of nodes in this example.

All the 2s come from the number of drives allowed to fail.

Using variables:

d :=  number of drives allowed to fail  (= 2 in the examples)

N := total number of nodes in pool (3, 4, 5, ...)

n := number of nodes allowed to fail (= 1 in the examples)

X :=  d * (N - n)  the number you asked for

things can be expressed in more general form:

4+2/2  becomes:   (X + d*n) / d = N            <-- always the number of nodes

4+2     becomes:    X + d*n        = N * d      <-- always the number of disks in a stripe group

Cheers

-- Peter

[Raj_la]

Nice got it - thank you so much ford & Peter.

[kipcranford]

>

> 4 Nodes with +2:1 protection how we can 6+2 equation derived , in this 6+2 i understand +2 but how we get the number 6.

> also 3rd row of 5 Nodes how we get number 8.

>

I just added a blog post (with illustrations!) that shows what OneFS does for layout in the event of small files and/or over-protected clusters (i.e. small cluster, high protection).  Hopefully the blog format allows for more detail than what can be conveyed here.  The post is at:

https://community.emc.com/blogs/kip_cranford/2016/08/09/onefs-protection-policy-vs-actual-protection...

[idioms]

---

4+2, 6+2, 8+2  and so on represent the numbers of disks in a stripe group.

4+2/2 is the OneFS "idiom" for (4+2)/2, which mathematically is 3, just the number of nodes in this example.

All the 2s come from the number of drives allowed to fail.

---

It was very helpful, thanks!