stripe or concatenate meta devs

In general, striped meta volumes will give you better performance than concatenated ones because they reside on more spindles. There maybe some exceptions. If you don't have enough drives for all the meta members to be on separate drives (wrapping) you might want to consider concatenated. If you plan to re-stripe many meta volumes again at the host, you should consider concatenated. If you are making very large RAID5 or 6 metas, and your workload is largely sequential, concatenated may also be a good choice.

Hope this helps.

Just to add 1.5 cents to Quincy's answer, EMC suggests striping twice.
If your host supports striping, you can set up striping at three different levels.

1) at host level
2) at metavolume level
3) at RAID level

Since you are already striping at RAID level, you have to choose between striping at host or at meta level. But please always remember what Quincy said.. Striped meta usually performs better then concat metavolumes. Unless you don't have enough drives in the backend.

Generally the Symmetrix performance group recommends striping not more than 2 levels. A single level of striping is good too. Too many levels and you get a plaid that any sane Scotsman wouldn't be seen dead in.

1 way or 2 way striping is good, avoid 3 way striping.
the 1st stripe is always in the array and it's upo to you where you'd want the 2nd stripe.

Wow thanks for the information guys. Based on that knowledge I will keep striping most of the presented luns, however I can think of a few large meta luns that do wrap around. In that case I will use concatenating metas.

HankDorsett: 3+1 is a raid 5 level stripe (3 disk stripe + hot spare) and our hypers span multiple 3+1 groups to form the 2nd stripe.


Quincy56 : You say that a stripe uses more spindles, I am trying to understand that. Say we create a meta with 6 hypers, wont they use the same pool of devices whether you stripe or concat? which will make them from the same number of spindles?

Is RAID 3+1 considered a stripe?

We stripe all of our meta devs and on a couple of hosts we use VXVM to stripe again.

Dave ...i think you have misunderstood what raid-5 3+1 is ..take a look at this link, specifically RAID-5 description.

http://en.wikipedia.org/wiki/Redundant_array_of_independent_disks



true, meta is composed of the same number of spindles but when it comes to actually reading and writing ..that where the difference comes in. For example a 6 (8 gigabyte) member meta build on top of RAID-5 3+1 will reside on 24 spindles (if there is no wrapping around). So now you present this 48G device to your host and start writing to it. If this is a concatenated meta and the host is writing to the beginning of the file system ..only those particular drives are involved, now if you are writing/reading to a striped meta ..all 24 spindles are involved.

Sorry, I should have been more clear. If I was doing concatenated metas, I would always create them on the same raid group, all on the same drives, spreading a concatenated meta across drives doesn't make much sense.

hi dynamox,
I understand what raid-5 is, however maybe i am confusing things.

This is my understanding, can you please tell me if i have it wrong.

A 8 gigabyte hyper spans 4 physical spindles.
Those spindles are configured in a raid-5 3+1. now the 3+1 refers to the 4 spindle striped group minus the parity, which referred to as the +1. So you only have the capacity of 3 disks but striped performance of 4.

Now the 8 gigabyte hyper spans multiples of these 3+1 groups to make each 8gig dev.

Sorry if that is confusing but I am just trying to understand this properly.

you got it ..now the 8 gigabyte hyper resides on 4 drives, here is output from "symdev show" for a RAID-5 devices, this is a 8.6G hyper

RAID-5 Device Information
    {
    Number of Tracks in a Stripe                : 4
    Overall Ready State of RAID-5 Device        : ReadyNoOtherMirror
    Overall WriteProtect State of RAID-5 Device : EnabledNoOtherMirror
    Member Number of the Failing Device         : None
    Member Number that Invoked the Spare        : None
    Disk Director (DA) that Owns the Spare      : Inv
    Copy Direction                              : N/A
    RAID-5 Hyper Devices (3+1):
        {
        Device : 0549
            {
            --------------------------------------------------------------
             Disk     DA       Hyper       Member    Spare       Disk
            DA :IT   Vol#   Num Cap(MB)  Num Status  Status  Grp#  Cap(MB)
            --------------------------------------------------------------
            01A:Cc    258    43    2878    3 RW      N/A        1   140014
            01B:Dc    648    43    2878    4 RW      N/A        1   140014
            16C:Dc    699    43    2878    1 RW      N/A        1   140014
            16D:Cc    258    43    2878    2 RW      N/A        1   140014
            }
        }
    }

A 3RAID5 logical device is created across 4 physical disks. A 7RAID5 device over 8 physical disks. 6RAID6 over 8 physical disks, and 14RAID6 over 16, with the RAID6 having 2 disks as overhead for parity in each group.

In Symmetrix, you can group individual logical volumes together to make a larger logical volume that is presented to the host and this is called a meta volume. The meta volume can be striped (at 1 cylinder for DMX3/4, 2 cylinders for DMX1/2 and below or 960k) or can be concatenated.

So if you were to take 2 3RAID5 devices and make them into a striped meta volume, the meta volume should reside on 8 disks, not just 4. However if you create a concatenated meta, I think it makes most sense to put both members on the same raid group. Think of this as being able to create a very large 3RAID5 volume without a meta. So in this case, the 2 member meta would reside on the same 4 drives. This is NOT ever recommended for a striped meta.

You are absolutely right .. expanding a concat meta means appending a new device at meta-tail. Expanding a striped metavolume requires a BCV since the real meta is first dissolved and later reformed with added members. Thus you need a place where to copy data while dissolving original meta. And later you use the BCV to stripe data back on all metamembers.

I have been pondering about this whole striping vs concatenating as well
correct me if I'm wrong here but it's a lot easier to add additional devices to a concatenated meta than a striped one right? last time I tried this in ECC it asked if I have BCVs for the striped meta I was about to expand.