How do you calculate usable capacity for Isilon?

to add to Mark's comment, one thing that you need to keep in mind that you should always have enough free capacity in the cluster to be able to smartfail out a node. For example i have a cluster that consists of 7x108NL nodes for total capacity of 698 Terabytes. I have to make sure that i always have at least ~100TB of free capacity to be able to smartfail a node out.

Here is the general accepted formula used when sizing:

1) Find total raw TB in base 10

2) Multiply that result by (1000^4/1024^4) to get base 2 TB

3) Subtract 1 GB per drive for the OS partitions

4) Subtract 0.0083% of that result to account for the filesystem format

5) Subtract the protection overhead from that result

As for the protection overhead that you are planning to use, look to the "OneFS User Guide" on support.emc.com.  Skip to the section: "OneFS data protection" where it will talk about N+M data protection, protection schemes such as N+1, N+2:1 (default), 2x, etc and the associated cost/parity overhead.  Also, you will see a very good matrix listing the percent overhead which begins by reminding us: "The parity overhead for each protection level depends on the file size and the number of nodes in the cluster."

OneFS User Guide now called OneFS 7.0.2 Administration Guide

You can refer chart below for capacity calculation..

Number of +1 overhead +2:1 overhead +2 overhead +3:1 overhead +3 overhead +4 overhead

nodes

3 nodes 2+1 (33%) 4+2 (33%) 3x 3+3 (50%) 3x 3x

4 nodes 3+1 (25%) 6+2 (25%) 2+2 (50%) 9+3 (25%) 4x 4x

5 nodes 4+1 (20%) 8+2 (20%) 3+2 (40%) 12+3 (20%) 4x 5x

6 nodes 5+1 (17%) 10+2 (17%) 4+2 (34%) 15+3 (17%) 3+3 (50%) 5x

7 nodes 6+1 (14%) 12+2 (14%) 5+2 (28%) 16+3 (15%) 4+3 (43%) 5x

8 nodes 7+1 (12.5%) 14+2 (12.5%) 6+2 (25%) 16+3 (15%) 5+3 (38%) 4+4 (50%)

9 nodes 8+1 (11%) 16+2 (11%) 7+2 (22%) 16+3 (15%) 6+3 (33%) 5+4 (44%)

10 nodes 10+1 (10%) 16+2 (11%) 8+2 (20%) 16+3 (15%) 7+3 (30%) 6+4 (40%)

12 nodes 11+1 (9%) 16+2 (11%) 10+2 (17%) 16+3 (15%) 9+3 (25%) 8+4 (33%)

14 nodes 13+1 (8%) 16+2 (11%) 12+2 (15%) 16+3 (15%) 11+3 (21%) 10+4 (29%)

16 nodes 15+1 (6%) 16+2 (11%) 14+2 (13%) 16+3 (15%) 13+3 (19%) 12+4 (25%)

18 nodes 16+1 (5%) 16+2 (11%) 16+2 (11%) 16+3 (15%) 15+3 (17%) 14+4 (22%)

20 nodes 16+1 (5%) 16+2 (11%) 16+2 (11%) 16+3 (15%) 16+3 (15%) 16+4 (20%)

30 nodes 16+1 (5%) 16+2 (11%) 16+2 (11%) 16+3 (15%) 16+3 (15%) 16+4 (20%)

Hi,

Can you please send the Isilon doc from where you pulled this chart for capacity RAW vs Usable? Using the on-line capacity calculations at this website.....

https://isilon-lawndart.herokuapp.com

the overhead  is much much higher than this.

Please help me explain to my customer......

Thanks!

The chart above is showing just the overhead associated with OneFS protection, for suitably "large" files where "large" means at least one complete stripe in size at any given cluster or node pool size.

The Isilon sizing tool is also showing you the overhead from protection, but also accounts for the "overhead" that comes from base-10/base-2 drive size conversion, space consumed by the filesystem itself, etc.  From the sizing tool FAQ:

-----

How is Usable Space calculated?

The amount of space, in base-2 terabytes, after factoring for OneFS protection overhead and OneFS filesystem overhead.

Here is the general formula for usable space:

- Find total raw TB in base 10

- Multiply that result by (1000^4/1024^4) to get base 2 TB

- Subtract 1 GB per drive for the OS partitions

- Subtract 0.0083% of that result to account for the filesystem format

- Subtract the protection overhead from that result

-----

The sizing of Isilon clusters is entirely dependent on the number of nodes, and is done per file, since we protect data per file with an Erasure Coding algorithm, not based upon a raid group or something similar.  So unfortunately you can't just say 20%.  With larger files and at N+2:1 protection you're looking at 33% overhead for 3 nodes, 25% for 4 nodes, 20% for 5 nodes, etc.  But with tiny files, the math is different.  Here is an easy way to understand it on a cluster.  Compare the output of:

du -sh /ifs/path

&

du -shA /ifs/path

That'll show you the difference between what windows would term size, and size on disk.  So including protection overhead or not.

With lots of data this takes a while to calculate, so pick a smaller subfolder to start like this:

isi01-1# du -sh /ifs/DMTEST/few_large_win

1.5T    /ifs/DMTEST/few_large_win

isi01-1# du -shA /ifs/DMTEST/few_large_win

1.0T    /ifs/DMTEST/few_large_win

This is an example of a dataset with small numbers of large files (100 x 10GB files).  This is on a 3 node cluster.  If the cluster were larger, we could stripe that existing data across more nodes, and have less total data stripes per FEC calculation.  Therefore the overhead as a percentage would go down.  If your manager wants to understand how to track usage over time for budgetary perposes, I would highly suggest looking at the latest version of InsightIQ (3.1), and/or some of the other tools available for monitoring and alerting / trending (SNMP, Nagios, Splunk).

Does this help?  There are some other threads that describe this in more detail in the last few weeks.

~Chris Klosterman

Senior Solution Architect

EMC Isilon Offer & Enablement Team

chris.klosterman@emc.com

twitter: @croaking

Also, if you have OneFS 7.2 and InsightIQ 3.1, it will show an estimate of how much more data you add to your cluster. It factors in the overhead of the files you are currently storing to estimate how much more logical data you can add.



No, no spreadsheet.

This logic is part of the Isilon Sizing Tool, not sure if you have access to that or not.  If you don't then the alternative is to take the pieces of logic from this thread (e.g. generic protection overhead, filesystem overhead, etc) and do the calculations manually...

--kip