OneFS Protection For Idiots

Hi,

I've been trying to get my head around OneFS protection and how it manages a file, breaks it into data units, assigns the correct FEC protection units, creates file stripes (or whatever the correct term is) and distributes this across a cluster and to be honest, my brain hurts.

Is there an idiots guide to this? Apologies for probably asking the same questions as others have in previous posts but after reading a few EMC documents, various notes and trawling the internet i'm slowly losing the will to live :)

Background

We added 3 nodes to an existing node pools so there are 7 in total. That pool did have a protection of 2d:1n but OneFs recommended it be changed to 3d:1n1d after the addition of these nodes. Since that we have seen a rapid decrease in available space in that pool which i can only explain as being caused by this change and the additional protection being put in place. I need to raise this increase in usage with our client but before i do i need to try get it straight in my head how OneFS Protection is working so can feel pretty confident in what i am talking about when i try explain it.

Answer

I have done some digging online and think i have the basic process. When a file is written by a client to a node is broken into 128Kb data stripe units up to a maximum of 16 data stripe units to form a file stripe. I believe protection stripe units (3 in our case if using 3d:1n1d protection) are then calculated for each file stripe and then the stripe is written across all nodes in the node pool.

Assuming i have this right where i am struggling is with a couple of things;

Each stripe unit is written to a single drive in a node isnt it? So how does that work in a 7 node cluster if potentially the file stripe could contain 16 data strip units and 3 FEC protection units? Just can't seem to get my head round how the data is actually distributed across the node pool.

And then i have looked at some of the actual  files using the isi get command and most files are showing with this protection 11+3/2. From what ive read the 11 is the number of data strip units, 3 is the FEC Protection Units and the last 2 is the number of stripe units allowed per node. But how can a load of files that are different sizes have the same number of data stripe units and where does this maximum number of strip units per node come from?

So i would really appreciate if someone could either help me understand the above or point be towards any documents/blogs that would explain it at an idiots level.