Doubling the frame rate typically doubles the size of the files (you are doubling the number of frames) if you are using the same codec and same resolution.
So if you are using 24TB currently (48TB cluster @ 50% utilization), doubling it means 48TB of video data. Add two more nodes for 72TB total and you are at 66% for the new workflow.
That's not necessarily the case, because as Isilon clusters grow, the overhead from data protection as a percentage, decreases. This is especially true with larger files like your video files. So to give you an example, if you keep the same protection at perhaps the default of N+2:1 on your 4 node cluster, a 100GB file might consume about 125GB of space, because the overhead is roughly 25%. Add 4 more nodes, and that same file when restriped as autobalance runs, now only consumes 112.5TB of space, because the overhad percentage drops to 12.5%. Matt's math was I'm sure to try and help you minimize the number of nodes that you have to add to the cluster, while keeping it less than say 80% or so full. If you want to stay at roughly 50% full, then yes double the node count.
Of course the other thing to keep in mind is that as a cluster grows, so does the number of moving parts and thus the likelihood of a failure. OneFS 7.2 clusters can auto-calculate the MTTDL and tell you if the protection level you're using is adequate or not based upon this. (note this feature has to be enabled after upgrade with 1 CLI command) It is on by default on new 7.2 clusters, just not on upgrades.
Hope this helps,
Chris Klosterman
Senior Solution Architect EMC Isilon Offer & Enablement Team
The limitation is "not exceed 70% raw capacity". I am confused with Matt's calculation... adding two more nodes give 72TB total, but how? Again how did he get 66%. If you understand, please explain, Chris...
Maybe this helps explain the efficiency of storage get higher with more nodes and thus less protection overhead (). This was taken from the "docu54199_OneFS-7.1.1-CLI-Administration-Guide.pdf".
mattashton1
93 Posts
0
January 23rd, 2015 16:00
Hi RRAO,
Doubling the frame rate typically doubles the size of the files (you are doubling the number of frames) if you are using the same codec and same resolution.
So if you are using 24TB currently (48TB cluster @ 50% utilization), doubling it means 48TB of video data. Add two more nodes for 72TB total and you are at 66% for the new workflow.
Make sense?
Cheers,
Matt
RRAO1
1 Rookie
•
121 Posts
0
January 24th, 2015 08:00
It should be 8 nodes total since it is doubling the frames from existing 48TB - 4 nodes.
crklosterman
450 Posts
1
January 24th, 2015 15:00
RRAO,
That's not necessarily the case, because as Isilon clusters grow, the overhead from data protection as a percentage, decreases. This is especially true with larger files like your video files. So to give you an example, if you keep the same protection at perhaps the default of N+2:1 on your 4 node cluster, a 100GB file might consume about 125GB of space, because the overhead is roughly 25%. Add 4 more nodes, and that same file when restriped as autobalance runs, now only consumes 112.5TB of space, because the overhad percentage drops to 12.5%. Matt's math was I'm sure to try and help you minimize the number of nodes that you have to add to the cluster, while keeping it less than say 80% or so full. If you want to stay at roughly 50% full, then yes double the node count.
Of course the other thing to keep in mind is that as a cluster grows, so does the number of moving parts and thus the likelihood of a failure. OneFS 7.2 clusters can auto-calculate the MTTDL and tell you if the protection level you're using is adequate or not based upon this. (note this feature has to be enabled after upgrade with 1 CLI command) It is on by default on new 7.2 clusters, just not on upgrades.
Hope this helps,
Chris Klosterman
Senior Solution Architect EMC Isilon Offer & Enablement Team
email: chris.klosterman@emc.com
twitter: @croaking
RRAO1
1 Rookie
•
121 Posts
0
January 24th, 2015 16:00
The limitation is "not exceed 70% raw capacity". I am confused with Matt's calculation... adding two more nodes give 72TB total, but how? Again how did he get 66%. If you understand, please explain, Chris...
Phil.Lam
3 Apprentice
•
631 Posts
0
January 25th, 2015 21:00
RRAO,
Maybe this helps explain the efficiency of storage get higher with more nodes and thus less protection overhead (). This was taken from the "docu54199_OneFS-7.1.1-CLI-Administration-Guide.pdf".
crklosterman
450 Posts
0
January 27th, 2015 12:00
Phil,
That is exactly the point I was trying to make, thank you. The overhead per file goes down as a cluster gets larger, given a fixed protection level.
Hope this helps,
Chris Klosterman
Senior Solution Architect EMC Isilon Offer & Enablement Team
email: chris.klosterman@emc.com
twitter: @croaking