Clariion CX3-20F maximum bandwidth - simple question

Your post has been moved to the Unified Storage community where it will likely find the correct audience.

In general, each backend bus can sustain 360MB/sec per Storage processor.  So on a CX3-20, there is 1 bus per SP and 2 SPs, 360MB/sec x 2 = 720MB/sec maximum throughput.  This backend throughput is not necessarily what you can get to the host though, it depends on several factors like processor performance in the SP and RAID Group/LUN Layouts.

Since each SP has an effective maximum bandwidth of 360MB/sec to the backend disk, the first thing you will need to do to exceed 360MB/sec on a CX3-20 is balance the load across both SPs.  Create two LUNs, one owned by each SP and present them to the host.  Then either configure the application to read/write to both LUNs simultaneously, or stripe a filesystem across both LUNs so that all reads are hitting both LUNs at the same time.  If you stripe across two LUNs, they need to be from different RAID Groups or you will have head thrashing.

Also, consider that the 6+1 Raid Group has a 384KB stripe, so you will want to try and get write IO sizes to 384KB or a multiple of 384KB.  If you stripe at the host level, then the effective stripe size is 768KB.   If the array is doing all large block, set the cache page size to 16KB as well. In general, if it's true sequential, any IO size that is a multiple of 64KB will be okay though.

To directly answer your question, here are the bus counts for CX3 and CX4 based EMC arrays..

CX3-20 / CX4-120 = 1 backend port per SP

CX3-40 / CX4-240 = 2 backend ports per SP

CX3-80 / CX4-480 = 4 backend ports per SP

CX4-960 = 4 or 8 backend ports per SP

Hope that helps.

First of all: a bus might have a max bandwidth of 360MBps, but that for sustained data throughput and I'm pretty sure that you wil nog achieve that in a random I/O environment. If you have a 180MBps throughput on 1 host, this most certainly is some sort of a backup server: large sequential blocks. Is this what you would use your CX3-20 for ?

But to answer your question: yes: if 1 host uses 180MBps, the other hosts have to fight for the remaining 180. But consider this: all I/Os are processed at the same time, it's not like you can reserve half the bandwidth for 1 host and share the rest for the other hosts.

The CX3-40 has 2 buses, but the current model with 2 buses is the CX4-240. The CX3 is a somewhat older model.

And: more cache is better, so if you really need a lot of performance, get more cache, so a larger CX4. More buses is good as well, but to absorb write peaks cache is the word. More buses can transport the data faster to the disks.

It's a mix of things and IT DEPENDS what you'll be using your Storage Array for.

Where did you read that a single disk can do 160MBps ? If you have enough cache and data to send to it and if the blocks are large enough you might get these kind of figures, but a rule of thumb is 10MBps in a random I/O environment. So a Raid Group in RAID5 (6+1) can do around 70MBps under normal workload. I've seen RGs on a CX3 perform up to 200MBps, but this was during LUN migrations in the CX3 internally. If you need to send 360MB per second to your Storage Array, this data has to come from somewhere ? Is your host capable of sending this much data ?

Thanks for the fast answers! Here is a 2nd iteration. I hope it will put an end to my struggles.

Please note: I'm optimizing for ***sequential*** throughput, not random.

Once again, on CX3-20F we have two Raid 5 groups 6+1, 6+1 of fast FC disks (sustained i/o from media up to 160MB/sec stated in Seagate spec), and in each RG our host is allocated about 850GB of contiguous space (2 volumes). All LUNs were owned by SPA. On Windows 2008 host I measured max 360 MB/sec sustained sequential read i/o total (about 180MB/sec for each raid group, i/o block size 1MB, 128KB was pretty close to that though). Subsequently I realized this is due to single bus.

After I provided feedback about low performance, this is what was done by our storage guys:

4 LUNs were created in each Raid group, and 2 hybrid(?) metaLUNs were created, each of 4 LUNs from both raid groups.

Raid group LUN MetaLUN  AssignedSP  CurrSP

1               11    100          A                  A

1               12    100          B                  A

1               13    200          A                  A

1               14    200          B                  A

2               21    100          A                  A

2               22    100          B                  A

2               23    200          A                  A

2               24    200          B                  A

So now host can see metaLUNs 100 and 200 as two volumes 850GB each. Note that all LUNs are all still owned by SPA. I run benchmarks and not only did I not see any improvement, I saw decrease vs what we had before.

Explanation: before we had two sequential i/o running in each of raid groups. Total was bottlenecked by single bus / single SP. Now while we still have single bus / single SP limitation, each of our host volumes accesses all the disks (due to metaLUNs), so when we run sequential i/o on each volume (metaLUN), to disks this is more like semi-random i/o, hence decrease in performance. Am I right?

Questions:

1. Is my understanding right that all LUNs in a metaLUN must be owned by the same SP? (in which case an effort to assign different SP's within a metaLUN like shown above was a waste of time).

2. Is my understanding right that since we optimize for sequential i/o and since we can't assign LUNs to different SP's in the same metaLUN there is no benefit to creating metaLUNs striping across raid groups, like they were created above?

2a. Let's say even if there was any benefit to metaLUNs and striping across raid groups, what could possibly be the benefit of creating and using 4 LUNs for each metaLUN instead of 2 (one in each RG)?

3. It looks to me that the best we can do to optimize sequential throughput on the host is to delete all 8 LUNs and crease one LUN in each RG, and then assign them to different SP's. I think this is exactly what was suggested. I do not want to do any striping in Windows. For a moment I thought metaLUN will be a substitute but now I realized I was wrong. There is also a factor here that our application is using mostly block sizes 64-256KB, so having more disks (larger stripe 768KB ) is not going to benefit us. Right? Instead it's best to split the disks and have stripe 384KB and have sequential i/o going at each of raid groups independently. Any feedback on my thoughts?

4. Trespassing. This may be related to our config or maybe not, but it looks like on our Windows 2008 host all SPB LUNs want to trespass to SPA, limiting us to 360MB/sec. The same trespassing occurs on another Windows host that is connected to the same array. Any gotchas here? Any specific Powerpath settings to set to make sure trespass doesn't happen or if it happens it is reversed automatically? I think we have all proper licenses and all recent firmware and so on. There are two HBAs and each shows two paths online. Maybe there is some doco that specifically addresses Windows trespassing problems?

Thanks a lot for any more insight!