Highlighted
Not applicable

HOW TO CALCULATE REQUIRED THROUGHPUT

Hi All

We are VFX studio and a currently looking for a 20TB Storage connected to 100 users.
Please guide us on the specs we should look in for while deciding a new storage device.

Below are the details of our current scenario

Current Fileserver

Server Model No: HP Prolient Ml110G6

Processor: Intel (R) XEON X3430 @2.40GHZ(Quad Core)

Memory: 8GB

OS: CentOS

Software Raid 1+0 : 4 X 1TB SAT HDD

Usable storage: 2TB

Data Shared through SAMBA share to 30 users

Dataflow:

All files reside on one Fileserver

User: Software read files from the fileserver then computes (renders) them locally and write back to the fileserver.

There are random read write requests

Typical File sizes:

  1. Average size of each frame - 12 MB per frame
  2. Number of frames per rendering node - 50 frames
  3. Average rendering time per frame - 45 sec

Network

30 Users connected to switch at 1Gbps

Fileserver connected to switch at 5Gbps (Link aggregation of 5 Ethernet ports)

We get 101 MBps throughput when all 30 nodes are rendering simultaneously

While rendering peak network usage per user node is about 500Mbps

While rendering Average Network usage per user node is about 300Mbps

Read Write ratio 
80 : 20

Current issues:

Current server is not able to handle the read write request when all 30 users are rendering thus slowing down the rendering process.

Server runs at 100% CPU

Average throughput achieved: 101 MBps  (808Mbps)

Latency goes upto 256 miliseconds

Thanks

9 Replies
Highlighted
7 Thorium

Re: HOW TO CALCULATE REQUIRED THROUGHPUT

have you talked to EMC or local reseller about Isilon ? They have solution that could possibly help you with your use case.

0 Kudos
Highlighted
Not applicable

Re: HOW TO CALCULATE REQUIRED THROUGHPUT

Hi

We have been discussing this with one of local EMC channel partners for couple of weeks but have not reached a justified solution / answer 

0 Kudos
Highlighted
7 Thorium

Re: HOW TO CALCULATE REQUIRED THROUGHPUT

they need to bring somebody who can talk Isilon,  maybe setup an POC. While you may not need the capacity offered by Isilon (i think it's up to 20 perabytes now), their s-series nodes offer some ridiculous IOPS.

0 Kudos
Highlighted
Not applicable

Re: HOW TO CALCULATE REQUIRED THROUGHPUT

they have suggested us VNXE3150;2XSPDPE;8GB;12X3.5DS;6 x 600GB, 15K SAS

0 Kudos
Highlighted
7 Thorium

Re: HOW TO CALCULATE REQUIRED THROUGHPUT

i am not very familiar with VNXe but look at its network configuration guide, there are only so many ports per storage process and i am not sure how many of them can be configured in LACP trunk.

0 Kudos
Highlighted
4 Beryllium

Re: HOW TO CALCULATE REQUIRED THROUGHPUT

With the four-port GbE module, the VNXe can do a four-port link aggregation.  Looking at the specs you have posted, I'm surprised they didn't mention Isilon. For such a small capacity, Isilon could be a nice fit. The VNXe config you list doesn't appear to have 20TB of capacity, unless your 12x disk pack is all 2TB SATA....

0 Kudos
Highlighted
4 Beryllium

Re: HOW TO CALCULATE REQUIRED THROUGHPUT

As recommended by others, my vote is also with Isilon. To assure you more, I would like to share my experience on one of the event I attended in EMC Forum 2012. We discussed, EMC Isilon and OneFS 7.0 back in 2012. To my surprise, there were many VFX Studios who found EMC Isilon the best solution for their business requirement! Moreover, one of them even said and I quote, may be EMC found Isilon recently but we've discovered Isilon before EMC did. (He was talking about before EMC took over Isilon) That guy is a CEO of leading VFX Studion in Bangalore, India!

So as Dynamox suggested, get a hold of someone from Local Team for a demo or POC on Isilon. It will give you more clarity on purchasing right solution for your business requirement. (I am almost sure when you compare VNXe and Isilon for your requirement you will go with Isilon! )

0 Kudos
Highlighted

Re: HOW TO CALCULATE REQUIRED THROUGHPUT

I too concur that Isilon is a great fit for VFX.  Many of the biggest movies are made on Isilon.  Not only does it fit the performance and capacity expected at the start of a project, it is extremely easy to expand.  You can even add more nodes while if full production with no down time.  Very cool technology in my opinion.

Highlighted
2 Bronze

Re: HOW TO CALCULATE REQUIRED THROUGHPUT

Hi,

I was wondering if you came to any solution with this? I too agree that Isilon is the best option. In fact your questions looks very much like several EMCTA questions from the exam.

If you haven't come to any solution, please say so as I could help with some Isilon sizing ideas.

Cheers.

0 Kudos