Just trying to decide between RAID10 (16 drives plus 1 hot spare) or RAID6 (16 drives plus 1 hot spare).  But I also have 3 Flash drives to use for Fast Cache.  I could certainly use the extra storage space that RAID6 would give me for future growth, but I am concerned about the write penalty.  However if Fast Cache can mitigate this penalty then maybe its OK to use RAID6?

Performance and Reliability is what I'm after... hence not interested in RAID5 anymore.

I have 17 SAS 10k 600GB 2.5" drives, 17 2TB NL-SAS 3.5" drives and 3 200GB Fast Cache 2.5" drives.

From our performance analysis sent to EMC, they said the RAID6 plus fast cache would be about 1000 IOPS higher than our current NX4 Celerra's 95th percentile.

So I was considering two 16 drive arrays.. just which raid level (6 or 10), with one hot spare per drive type (hence the 17 drives for each).

OR, 4 8 drive arrays (two from SAS and two from NL-SAS).  Though I like the idea of a higher spindle count for performance.  I heard that it is very important not to break up the spindles. If you separate them then you have no means by which to leverage the combined IOPS and instead of getting the advantage of consolidation you are putting the IOPS into silos so that when one RAID pair is idle and another is busy they have no way to load balance. Think of it like your CPUs. You don't carve out the CPUs for just one VM and another CPU for another VM. You want all idle CPU power available to any VM that wants to use it so that you are getting maximum performance with minimum waste. With one big array, pretty much every disk operation is faster than when they are split. And because disk operations happen more quickly it also keeps queue lengths low making it even more likely that the next request will enter an empty queue. With split drives you are almost guaranteed to have some idle while others are getting beat to death. It also helps to spread out wear and tear to give you more time between hardware failures.

What do you guys think?