This post is more than 5 years old
50 Posts
2
3029
FAST VP SATA usage and relocation rate
So we are FINALLY cruising along in our VMAX 20K testing but i'm a little concerned with FAST VP SATA usage. I’ve been testing a pretty heavy SQL workload over the past week from two servers in FAST VP. If I run both at the same time it will peak to about 48K IOPS. In our configuration we have 16 EFD in R5, 388 10K FC in R5, and 280 SATA in R6. The SATA is broken up into two disk groups due to using some for the vault, but both SATA disk groups are in the same pool. My concern is when I put a storage group in a policy that allows 100% on any tier it doesn’t seem to use FC much at all, although it does put a small percentage in EFD. In fact I moved a SG entirely to FC and FAST VP demoted down to SATA pretty quickly even while I was running jobs. I also tried binding to FC since the SG's were origionally binded to SATA but with the “Allocate by FAST policy” enabled in 5876 it will put new writes to the tier that it thinks it should where in 5875 all new writes went to whatever teir it was binded to. My assumption from reading "Implementing Fully Automated Storage Tiering for Virtual Pools (FAST VP) for EMC Symmetrix VMAX Series Arrays Technical Notes" would be I/O in FC should be higher than SATA I’m seeing 6K IOPS in two SATA Disk Groups for a total of 12K in SATA and only 600 IOPS in the FC. SATA seems to be handling it fine now, but we only have 24TB out of 510TB allocated so I’m worried what will happen with this SATA tier once we go live and start migrating a bunch of hosts. Also with SATA being R6 having a lot of I/O there really hits the director CPUs due to the dual parity. With only two hosts and this I/O i've seen the directors average 55-65% utilization. Hosts are using one FA on each director for a total of 8. Also the two hosts that are on here are not sharing an FA. One server is on all E0 ports and the other is on all F0 ports. Has anyone seen this SATA utilization at all in their environment and have any thoughts?
Also, what is everyones thoughts on the relocation rate? Keep it conservative at 7 or more aggressive?
Thanks
Quincy561
1.3K Posts
0
August 28th, 2012 10:00
I'm not sure IO generated that way will have a skew of active vs. inactive data.
RAID1 for the FC tier as a best practice has been documented many places. This tier is a IOPs tier. You want the cheapest IOPs/$$ and RAID1 will give you that. RAID5 will cost more per IOP, as long as there are writes in the workload.
FAST VP doesn't take RAID protection into account, but it does look at utilization of the tiers/drives and won't promote onto overloaded tiers.
Quincy561
1.3K Posts
1
August 28th, 2012 09:00
How are you generating IO? If you are using SQLIO, there is no skew. FAST VP requires some skew to be effective.
Also AFAIK the new allocation by policy will still prefer the bound tier.
And the FC tier should be RAID1 by best practice. This is a IO tier, not a capacity tier, so RAID1 is cheaper for IO than RAID5.
I would set the relocation rate to 1 if you want FAST VP to react more quickly in a test environment.
gblack2
50 Posts
0
August 28th, 2012 09:00
We are generating I/O with a full SQL run from bulk import, index, etc from a sql backup so it's real data.
I have to look to see where it's documented but my EMC TSG saw where the Allocation by Policy changed from 5875 to 5876. In 5875 it always put writes to the bounded teir until that teir was full.
Is the Raid1 for FC documented anywhere? I'm not sure if it matters in this case as from what i've read FAST VP doesn't take RAID level or drive speed into account. From what I understand the only thing it takes into account is drive type (EFD, FC (includes SAS), and SATA). The RAID1 best practice is definitely interesting.
We want to test as close to real world as we can. I assume the worry for high relocation rate that it will impact host i/o? What values do you typically see in the real world.
Thanks again
Quincy561
1.3K Posts
0
August 28th, 2012 10:00
Sorry about the duplicate posts, but I was not used to having my post moderated.
gblack2
50 Posts
0
August 28th, 2012 10:00
No problem I appreciate the replies. I'm going to do a test on FC and EFD opposed to FC, EFD, and SATA. If your response time theory is correct then I should see higher response times during this run. I just can't see why FAST VP would choose SATA in R6 over FC R5 when the FC tier also has a lower busy %.
Quincy561
1.3K Posts
1
August 28th, 2012 10:00
I'm not sure if your SQL load created that way will really cause a skew of active vs. inactive data.
The RAID1 on the FC tier is the best practice, documented many places. It is not required. If you use RAID5 you may need more FC drives to handle the workload. Remember this is an IOPs tier, you want the most IOPs/$$, not the most capacity per $$.
FAST VP does not look at protection, but it will look at response times, and not use overloaded tiers for promotion.
Higher relocation rates may impact response times somewhat, but it sounds like you are not really focusing on the response times at this point.
gblack2
50 Posts
0
August 28th, 2012 11:00
I'm not finding any documents that state the best practice is to have FC as Raid1 in a fast vp configuration. I'm not saying I don't agree with you, but our proposed solution from EMC was Raid5 so i'd like to find a document that states this before I bring it up as a potential issue. Do you know where I could find this?
Thanks
sauravrohilla
859 Posts
0
August 28th, 2012 20:00
Hmm, even we were told to have FC as raid 5 and Sata tier as raid 6 in our fast vp setup and we do see alot of movement across SATA and FC.
regards,
Saurabh
gblack2
50 Posts
0
August 29th, 2012 07:00
I found the doc that you were refering to (or one of them).
" For the FC tier, RAID 1 is the recommended protection level. Mirrored data devices on FC pools provide a higher level of performance than both RAID 5 and RAID 6, particularly for write workload. Availability of RAID 1, in regard to a dual-drive failure, is also greater than RAID 5. To obtain the best availability numbers for RAID 1 on FC, the use of lower capacity drives is recommended. "
FAST VP for EMC® Symmetrix® VMAX®
Theory and Best Practices
for Planning and Performance
Technical Notes
P/N 300-012-014
REV A04
June 2012
https://powerlink.emc.com/nsepn/webapps/btg548664833igtcuup4826/km/live1/en_US/Offering_Technical/Technical_Documentation/300-012-014.pdf?mtcs=ZXZlbnRUeXBlPUttQ2xpY2tTZWFyY2hSZXN1bHRzRXZlbnQsZG9jdW1lbnRJZD0wOTAxNDA2NjgwNWMwNWQ4LGRhdGFTb3VyY2U9RENUTV9lbl9VU18w
gblack2
50 Posts
0
August 31st, 2012 09:00
Apprently the RAID 1 best practice is something recent and there is some debate over it since 25% more capacity is lost over RAID 5. We're still in the debate but we may split the FC into RAID 5 and RAID 1, but wide striping both RAID 5 and RAID 1 over all of the FC disks. Unfortunately along with increasing the price per GB, it's going to complicate our FAST VP Tiers since you can't have different raid types in the same tier.
Since you can only have 3 tiers in a policy we won't be able to have a policy with all tiers. Our tiers would be EFD, FC_R1, FC_R5, and SATA_R6.
Thoughts?
Quincy561
1.3K Posts
1
August 31st, 2012 10:00
Yes, you can put enough EFD in to handle ALL the active workload, but that would be very expensive.
In most cases, we need the FC tier to handle up to 50% of the active IO. Again RAID1 will generally be cheaper per IOP than RAID5, and more reliable. This is an IOP tier, not a capacity one. Each write generates twice the IOPs on the disks vs. RAID1. And there is XOR overhead. That taken into account, RAID5 is generally about 50% MORE expensive for given IO load than RAID1 for the same quantity of disks.
We generally recommend RAID5 for the EFDs as they can deal with the extra IOs better, and we need the capacity there more than the FC tier as the FC tier generally has a lot more capacity.
Govindagouda
85 Posts
0
August 31st, 2012 10:00
Hi Garret,
Even we are moved from Raid 10 to Raid 5. and Sata on Raid 6. With FAST VP in Place the extra performance would be taken by SSD.
Govindagouda
85 Posts
0
August 31st, 2012 10:00
As I see that most of the Peopel are moving from Raid 10 to Raid 5 with Fast VP. We are one example of this implementations. We had Raid10 with 120TB of Oracle Database Capacity and now we have moved to Raid 5 with 32 EFD in place.
Again as said by Quincy Raid10 is non commentable.
Quincy561
1.3K Posts
0
August 31st, 2012 10:00
RAID1 for the FC pool was never a debate in the performance group.
Again you want the best IOPS/$ not the best GB/$ in the FC tier. If you can't use the capacity because it is overloaded with IO, the extra capacity you gain with RAID5 will be wasted. Also you must consider the availability of the pool where RAID1 will be much better.
gblack2
50 Posts
0
October 12th, 2012 12:00
According to our EMC SE it came out as best practice to do FC R1 right around the last EMC World. He said it turned into quite the religous debate between the EMC engineers and the SE's out in the field but ultimitely the new best practice for FC when using FAST VP is R1. We just went production with R5 (3+1) for EFD, R1 for FC, and R6 (6+2) for SATA. Just as Quincy explained, it's due to FC being used as an IO tier.