MD3000i Mixing drive sizes ?

If you mix drive sizes, the disk group space is capped by the smallest drive size. So, that 250GB will not be usable on the 1TB drives.

Your choices are:

- 14-disk raid 5 (yielding 13 x 750GB = 9750GB)

- 7-disk raid 5 with 750GB drives + 7-disk raid 5 with 1TB drives (yielding 6 x 750 + 6 x 1TB = 4500 + 6000 = 10500GB)

These numbers are obviously not accounting for the decimal to binary conversion.

Bummer about the lowest drive size.   I had hoped it would show the rest as available.

I must of errored in my math when I did it.   As you laid it out so clearly...   Actually,  it comes out the same.   An error in your math.  Should of been 14 x 750 which also equals 10,500GB right?

There probably would be some advantages to having it in two groups.  One of them would have to run pretty many VM's.  Like 40, all from one host though.  This is for our redundant DR site.   We have been running two MD3000i's down there,  but we have decided to consolidate it to just one. 

On a side note, any issues running an ESX host and a Windows host on the same MD3000i?  We just have not done it yet, is why I ask.  I can't see why it could possibly matter though.

Somehow I had read that you had 7 1TB drives and 7 750GB drives. If you opt to skip a hotspare (with 15 drives in an enclosure I'd recommend to allocate 1 drive as a hotspare because when a drive fails the performance hit is pretty significant on such a large raid 5), you would indeed have an extra 750GB if you go with a single raid 5, or an extra 1TB if you go with an 8-disk raid 5 and a 7-disk raid 5.

I struggled with this over night.   I actually come out best all the way around by doing a 14 drive RAID 6 with a host spare.  Gives me the most space with the most redundancy with the most performance. (IOPS)

It was hard for me to believe that I could throw out 250 gig per drive over 7 drives and this still be the case.  I'm a little tired but I checked my math a bunch of times.  I gotta be able to run 40 VM's or so on this thing.

You guys keep spewing this stuff out but our testing shows completely different on this.  You reduce your iops so much when you reduce your spindle count  for RAID 10 to really be better you have to increase your spindle count.

7 spindles sucks on IOPS.   As a matter of what we see, the 14 drives will never produce less write iops than 7 spindles of raid 10 even in RAID 6 config at it's worst.

We have several of these MD3000i's and the kiss of death is low spindle counts as far as performance goes.     I'm not sure how many spindles in RAID 10 would equal the write IOPS of a RAID 5, I am sure it would be require more.   A raid5 14 spindle count blows a 14 drives raid 10 totally out of the water on writes.  Raid 6 is about equal on the writes average.   We won't even go there on reads where the advantage is way way worse.

"If you want to run that many VMs on 7200rpm drives, you'll want to go with a 14-disk raid 10. If you cannot afford that, I'd suggest to go with something like 2 x 5-disk raid 5 and a 4-disk raid 5 to keep the parity calculation performance hit in check a bit more."

Remember that a 14 disk riad 10 only gives you 7 spindles of IOPS.  Your proposed config would be bad from our testing. We have tested it and retested it on these MD3000i's.    The 2TB limit has never become an issue for us.  But your right, set it up as you have suggested and there is no ay you could run 40 VMs on it without serious performance issues.   This particular group, the VM's are mosly XP VM's.  

I can only assume, maybe with other sans you may be correct,  but not with the MD3000i, spindle count is king.   The goal is to get the spindle couny high enough that controller saturation becomes the bottle neck.  14 drives seems to be good enough to achieve this.

If you have a dell lab with an MD3000i and an ESX host with some serious exchange, SQL, and flat file servers on it and can load them up?  Do some actual load testing your self on this.   

It was hard for me to believe that I could throw out 250 gig per drive over 7 drives and this still be the case.  I'm a little tired but I checked my math a bunch of times.  I gotta be able to run 40 VM's or so on this thing.

You never mentioned your intended use till this last post. This changes everything.

Space wise, yes you may be able to fit 40 VMs on there, but performance wise there's no way you can run 40 (production) VMs on a 14-disk raid 6.

If you want to run that many VMs on 7200rpm drives, you'll want to go with a 14-disk raid 10. If you cannot afford that, I'd suggest to go with something like 2 x 5-disk raid 5 and a 4-disk raid 5 to keep the parity calculation performance hit in check a bit more.

Also; you didn't mention if you're going with ESX/ESXi, a hyper-v-based solution, or something else. VMware's ESX/ESXi cannot use (virtual) disks over 2048GB or larger. They need to be "2048GB minus 512 bytes" or smaller.

Ok..  I think I came off a little rude.  Sorry about that.  I worked an all nighter last night and I think I am cranky.

In practical examples in testing.....

Our Exchange server DB information store runs close to 100gb.            Using SATA drives, with no other VM's running at all on a given ESX host the information store fails to load on boot if we have anything less than 8 drives in a group on these MD3000i's.   We can load it after boot manually, but on boot, never.  This is true of an 8 disk raid 10 as well.     At 8 drives in a RAID 5 or 14 disks in a raid 10 it loads on boot, but only if it is the 2nd VM we load after the primary domain controller.  In other words we can't even reboot the exchange vm without having to then manually load the information store after booting and logging into the VM once the enterprise gets beyond 1 or 2 vms.  At 14 disk raid 5 we found we can have full freadom not only on this note, but we can support 24 really busy, heavy server VM's with excellent performance. (I actually think we ran 48 of these VM's once during a host issue we were having and still had decent performance to the users but we could tell it was stressed. These are a mix of SQL, Imaging, Exchange, FAX, file server, and Vcenter VM's.  In RAID 10 with those same disks?   Performance just stinks when we throw the same load on it, no other way to put it and we are back to the Exchange information store issue which is really irritating.

I dunno, you tell me what conclusions you would draw from this?

Now one thing we do see easily with the MD3000i are iSCSI reservation issues, if we try to put two heavy VM's on the same lun we qucikly run into issues.  But when we divide the large disk group into more luns we never have issues with that.   iSCSI reservation issues are often mistaken for drive lock from what I have seen over the years.

I always forget something darn it.   There will be two hosts attached to this group.  Both are 2970's, 12 cores, 32GB ram.  They are currently attached to a 14 drive raid 5 group with 500gb 5400rpm sata drives and I am out of space.

This gets back to more questions?  We are going to try to hook the MD1000 up with the eight 1tb drives and seven 750gb drives to the MD3000i, copy all the VD's to it.  Export the 14 drive 500gb SATA drives disk group off the array and remove all the drives,  Export the seven 1tb drives and seven 750gb drives disk group, then move all the drives from the MD1000 enclosure to the MD3000i enclosure and then import the group back into the MD3000i.

Think it will work?  :)

I hope so,  we are talking about TB's and TB's and TB's of data here.  So much so?  No way to do it within a reasonable amount of time with restores.  Only block level transfers can handle this change.    You say I could of just added the MD1000 and been done with it right?    Well, as it turns out our backup to disk based backup vault is also out of space which is also serviced by an MD3000i so I am going to put the 500GB drives back in the MD1000 and attach it permanently to the dedicated backup server MD3000i san doubling it's storage capacity.

Just to give a clearer picture of what we intend to run on this group?

One Vcenter server VM, One Backup Domain controller / primary DNS server VM, One Webserver VM, one archive fax server VM, One flat File Server Vm, 30-ish XP VM's to support user workstation use.

But it will also serve as our test environement where we can bring up test copies of all or part of our enterprise VM's for implementation testing.

This is why I need the space.   The more the merrier.   I was hoping I could do the large RAID6 group and then have the rest of the space on the 1TB drives available as well.   I would of considered breaking up the group if it meant I did not lose as much space but that did not turn out to be the case when fully redundant raid levels are involved.

Right now most of the above is running on a 14 drive group using 500gb sata drives and I am out of space.   We did keep that group on raid 5 though.  The production heavy server VM's large groups have since been converted to raid 6 and our write IOPS did drop some compared to raid 5 but the trade off for the extra redundancy was worth it.   Even though we see it, I don't think the users feel to much difference at all truth be told.

I believe that your method should work. Exporting the disks from the expansion enclosure and re-importing them to the original enclosure would be the safest approach.

On the whole, the issue that you are seeing seems to be linked specifically to the SATA disks being used. In general, performance improves with spindle count, but it falls as the number of disks involved in parity calculation increases. Also, when feasible, it is recommended to create separate disk groups for high activity LUNs so that activity on one set of LUNs will only impact the disk group they are part of and not all the disks. So 2 disk groups should have overall better performance than a single large one.

However, as we know, performance has so many variables that it is tough to determine the exact single cause of poor performance. As the array tuning whitepaper says, apply config, test, tweak!

Thanks for the help to both of you.   I have decided to leave it on RAID5,  750gb drives are right on the edge of this for me.   If the enclosure was all 1TB drives I'd use raid 6.   At leats this gives me an additional 750gb of space.   :)