This post is more than 5 years old
121 Posts
0
2462
Isilon node calculation
Need to upgrade 4 node X200 48 TB cluster with 50% utilization of video. Want to upgrade from 15 FPS (NTSC) in H.264 codec with 15 days retention policy, 30 FPS (NTSC) the same codec with the same days retention policy... How many nodes of the same type will need to add and not exceed 70% raw capacity?
I don't know how to calculate and could not find any pages that explain the course. Please help.
mattashton1
93 Posts
0
January 23rd, 2015 16:00
Hi RRAO,
Doubling the frame rate typically doubles the size of the files (you are doubling the number of frames) if you are using the same codec and same resolution.
So if you are using 24TB currently (48TB cluster @ 50% utilization), doubling it means 48TB of video data. Add two more nodes for 72TB total and you are at 66% for the new workflow.
Make sense?
Cheers,
Matt
RRAO1
121 Posts
0
January 24th, 2015 08:00
It should be 8 nodes total since it is doubling the frames from existing 48TB - 4 nodes.
crklosterman
450 Posts
1
January 24th, 2015 15:00
RRAO,
That's not necessarily the case, because as Isilon clusters grow, the overhead from data protection as a percentage, decreases. This is especially true with larger files like your video files. So to give you an example, if you keep the same protection at perhaps the default of N+2:1 on your 4 node cluster, a 100GB file might consume about 125GB of space, because the overhead is roughly 25%. Add 4 more nodes, and that same file when restriped as autobalance runs, now only consumes 112.5TB of space, because the overhad percentage drops to 12.5%. Matt's math was I'm sure to try and help you minimize the number of nodes that you have to add to the cluster, while keeping it less than say 80% or so full. If you want to stay at roughly 50% full, then yes double the node count.
Of course the other thing to keep in mind is that as a cluster grows, so does the number of moving parts and thus the likelihood of a failure. OneFS 7.2 clusters can auto-calculate the MTTDL and tell you if the protection level you're using is adequate or not based upon this. (note this feature has to be enabled after upgrade with 1 CLI command) It is on by default on new 7.2 clusters, just not on upgrades.
Hope this helps,
Chris Klosterman
Senior Solution Architect EMC Isilon Offer & Enablement Team
email: chris.klosterman@emc.com
twitter: @croaking
RRAO1
121 Posts
0
January 24th, 2015 16:00
The limitation is "not exceed 70% raw capacity". I am confused with Matt's calculation... adding two more nodes give 72TB total, but how? Again how did he get 66%. If you understand, please explain, Chris...
Anonymous
5 Practitioner
5 Practitioner
•
274.2K Posts
1
January 24th, 2015 19:00
There is another aspect to consider - the node pool. If you add nodes with exactly the same config, they share the same node pool and protection overhead. I.e. Adding 2 nodes give you 2 nodes worth of usable space if protection is the default +2:1.
If you add different config, you need at least 3 nodes of them and the have their own protection overhead.
I.e. Adding 3 nodes give you 2 nodes worth of usable space if protection is the default +2:1.
Sent from my Mobile phone.
Phil.Lam
3 Apprentice
3 Apprentice
•
592 Posts
0
January 25th, 2015 21:00
RRAO,
Maybe this helps explain the efficiency of storage get higher with more nodes and thus less protection overhead (). This was taken from the "docu54199_OneFS-7.1.1-CLI-Administration-Guide.pdf".
crklosterman
450 Posts
0
January 27th, 2015 12:00
Phil,
That is exactly the point I was trying to make, thank you. The overhead per file goes down as a cluster gets larger, given a fixed protection level.
Hope this helps,
Chris Klosterman
Senior Solution Architect EMC Isilon Offer & Enablement Team
email: chris.klosterman@emc.com
twitter: @croaking