Unsolved
This post is more than 5 years old
34 Posts
0
678
June 24th, 2009 09:00
RAID Configuration Discussion
Hello and please bear with me through my very first post. I'll start by saying that I have already made the decision and started modifying my RGs and LUNs accordingly.
Nonetheless, having read through many of these posts over the last couple of days, I thought I would query the group for their thoughts/opinions to the decision I was faced with a few weeks ago.
We have a CX500 with 26 installed to read some newer drives for a shelf we recently added. The CX500 is currently used exclusively for backups. I have 7x2TB MetaLUNs presented to a direct attached host. Here was my original configuration:
Each enclosure/shelf has 2x7 Disk RGs as RAID5 (6+1). From each RG, I stripe a MetaLUN to its neighboring LUN (same shelf) and then present 2 TB to the 32-bit OS. Each shelf has enough raw to do this twice with about 512+ GB remaining from each RG. The MetaLUN for each drive letter to the OS has been assigned to the processors with an alternating strategy (SPA owns E,G,I while SPB owns F,H,J, etc). None of the drive letters to the OS have any more importance than the others. The application chooses the storage path by the "least used".
Being a man of symmetry, this seemed to be the best way to equally distribute processes and resources.
On Sunday afternoon, I kickoff an image duplication that reads from one single drive and copies the image to an LTO3 drive. This jobs runs for about 60 hours. Early Monday morning, I have a second duplication job that fires off, which on some images copies from the same drive but most often from another. This duplication takes about 6 hours. These duplication jobs inevitably run at the same time and will often be reading from the same MetaLUN or pair of RGs on the same shelf.
The decision that I made was to go to a RAID3 configuration instead of RAID5. With the RAID3 I could not do two 7 disk RGs, yet a 5 disk RG and 9 disk RG and my MetaLUNs have to be concatenated as opposed to striped (I think I have a good grasp on why). This required me to create my LUNs what seams a
little "off balance", at least as I see it in my head. This is the math I used: 5/14 = ?/2048 == 732 (rounded up) and 9/14 = ?/2048 == 1317.
My question for this forum is simply, "Did I make the right decision(s)?" Honestly, I'm just trying to get a discussion started to help support the strategy that I am using.
Thank you for your time.
Bart Perrier
Nonetheless, having read through many of these posts over the last couple of days, I thought I would query the group for their thoughts/opinions to the decision I was faced with a few weeks ago.
We have a CX500 with 26 installed to read some newer drives for a shelf we recently added. The CX500 is currently used exclusively for backups. I have 7x2TB MetaLUNs presented to a direct attached host. Here was my original configuration:
Each enclosure/shelf has 2x7 Disk RGs as RAID5 (6+1). From each RG, I stripe a MetaLUN to its neighboring LUN (same shelf) and then present 2 TB to the 32-bit OS. Each shelf has enough raw to do this twice with about 512+ GB remaining from each RG. The MetaLUN for each drive letter to the OS has been assigned to the processors with an alternating strategy (SPA owns E,G,I while SPB owns F,H,J, etc). None of the drive letters to the OS have any more importance than the others. The application chooses the storage path by the "least used".
Being a man of symmetry, this seemed to be the best way to equally distribute processes and resources.
On Sunday afternoon, I kickoff an image duplication that reads from one single drive and copies the image to an LTO3 drive. This jobs runs for about 60 hours. Early Monday morning, I have a second duplication job that fires off, which on some images copies from the same drive but most often from another. This duplication takes about 6 hours. These duplication jobs inevitably run at the same time and will often be reading from the same MetaLUN or pair of RGs on the same shelf.
The decision that I made was to go to a RAID3 configuration instead of RAID5. With the RAID3 I could not do two 7 disk RGs, yet a 5 disk RG and 9 disk RG and my MetaLUNs have to be concatenated as opposed to striped (I think I have a good grasp on why). This required me to create my LUNs what seams a
little "off balance", at least as I see it in my head. This is the math I used: 5/14 = ?/2048 == 732 (rounded up) and 9/14 = ?/2048 == 1317.
My question for this forum is simply, "Did I make the right decision(s)?" Honestly, I'm just trying to get a discussion started to help support the strategy that I am using.
Thank you for your time.
Bart Perrier
jps00
2 Intern
•
392 Posts
1
June 24th, 2009 10:00
Using RAID 3 for backup-to-disk is a recommended usage. RAID 3 has good, large-block sequential performance, particularly for reads. The (4+1) and (8+1) sized RAID 3 groups are recommended. Limit the backup streams to five or less per LUN.
Having written that, the threading of the backup application becomes important. If the application is single threaded, RAID 3 is the correct decision. If it is multi-threaded, RAID 5 is better.
With RAID 5 making the underlying RAID groups of the LUNs the same size is advisable. Same-sized RAID groups have the same service times. This is particularly beneficial when the LUNs are ganged together into MetaLUNs and bypassing cache. Same sizing avoids creating ¿hot-spots¿ out of the smaller RAID groups. Hot spots are RAID groups which will have a higher utilization than their peers. Assuming multiple threads, the smaller RAID groups will have a higher queue depth than their larger peers. This may result in different service times between the LUNs within the MetaLUN. (It is good to have them all about the same.)
A CX500 has two backend buses (two loops per SP). It would be advisable to vertically provision the underlying RAID 3 groups of their MetaLUN. (You are horizontally provisioned.) That is, alternately place each RAID group on a separate backend bus in round-robin fashion. This will avoid any RAID 3 read bottleneck.
RAID group selection and provisioning is discussed in detail in the ¿EMC CLARiiON Best Practices for Performance and Availability, FLARE Revision 28.5¿ in the 'RAID Performance characteristics' section. I note you are at rev. 26, however the 28.5 Best Practices recommendation is the same and includes additional information. In addition, there is more general information on threading, vertical provisioning, hot spots, and MetaLUNs found in the ¿EMC CLARiiON Storage System Fundamentals for Performance and Availability¿. Both of these documents are available on PowerLink.
Message was edited by: jps00 "(4+1) and (8+1)" correction.
jps00
roamdeep
34 Posts
0
June 24th, 2009 12:00
This application, NetBackup, allows multiple backup jobs to occur at the same time. Fortunately, my current settings prevent an excessive number of jobs from running to the same MetaLUN. This is what i have referred to as multiple threads, but right now, I'm a little hesitant to say so.
In your reply you mentioned the recommended RGs for RAID3 -- 3+1 and 8+1. I have only read about configuring a 4+1 or 8+1. Just checking so I make sure I am not missing a critical piece that will impact my performance.
Additionally, I have the documents your recommended and will be reading them over the next few days (weeks). But, with regards to veritcal provisioning, are you referring to a MetaLUN that has a component from B0-E1 and B0-E2 or B0-E1 and B1-E1? Maybe I'm way off on that but over the years, I have stayed away from configuring MetaLUNs across shelves.
Again, thanks for the reply and the nudge in the right direction.
Bart Perrier
RRR
4 Operator
•
5.7K Posts
0
June 25th, 2009 02:00
jps00
2 Intern
•
392 Posts
0
June 25th, 2009 05:00
'Bus balancing' of RAID groups is a lengthy discussion. Please refer to the documents I referred to. They contain a lot of information on the subject and 'how to do it'.
roamdeep
34 Posts
0
June 25th, 2009 06:00
I am into those documents and i have found a section in performance doc that discusses a RG across shelves (w/Fig. 16). A difference it shows from what I have been thinking is the RG across the shelves as opposed to the MetaLUN across the shelves. I may be in the position to do some testing with my environment to see what it looks like.
Thanks again for your help, jps.