This post is more than 5 years old
61 Posts
0
4288
Benefit of Meta Volume Member Count (greater than disk count)
When creating a meta volume you have the option to pick your:
"Meta Volume Member Count Including Head"
We have 24 disks in our SAS pool as you can tell from the screen shot. I see that 64 meta volumes is recommended and that I can actually select more meta volumes than I have actual disks.
Is there actually any performance benefits to adding more meta volumes than there are actual disks? You're still limited by spindles.
Thank you kindly,
Steven
seancummins
226 Posts
0
May 29th, 2013 09:00
Steven,
You don't need a direct correlation between front-end path/port count and meta member count; I was just pointing out that these two variables have a multiplying effect when it comes to the number of concurrent queues available for a particular host's workload. For example, if you have an 8-way striped meta mapped through four FA ports, then there are 4*8=32 queues that can concurrently service the workload on that meta.
The RR change from 1000 to 1 is also related to optimal use of these queues. With the default of 1000, the ESX NMP policy will send a stream of 1000 IOs to one FA port before switching paths -- the other FA paths, which have their own discrete CPU cores and queues on the VMAX, will be sitting idle while ESX directs all IO to a single path. This can result in longer queues on the currently active path, which in turn results in elongated response time (the deeper the queue, the higher the response time). By setting RR to 1, ESX switches to a different path for each IO, so the IO is balanced across all VMAX front-end CPU cores and queues.
You can think of a queue as being like a teller line at a bank -- in a particular line/queue, only one person is being serviced at a time. The time it takes for the "active" person's service to complete is called the Service Time. Response Time is the perceived amount of time it takes for a new person (or IO) to be serviced, starting at the time they enter back of the line. Response Time is the Service Time (amount of time it takes to complete one order/IO) multiplied by the queue depth (how many people are in front of you waiting to be serviced). Ideally you want to distribute your workload evenly among many queues, so the lines are short and response times are kept low.
Hope that helps,
- Sean
Zikas
278 Posts
1
May 28th, 2013 11:00
Hi smaiello,
what is going to be the use of that devices?
I mean Boot devices, REDO logs, data files, archives logs etc.
they will be used on what O.S. and with what Volume Manager?
smaiello
61 Posts
0
May 28th, 2013 12:00
These will be VMware VMFS datastores. We are striving to keep the number of VMs per data store to a limit of 15 VMs per volume.
Also thank you personally NY Yankees, you've been very kind and helpful and it's greatly appreciated. I'm totally new to the EMC world and haven't had the chance to attend any training yet. There's so much to learn about an enterprise storage array.
Thank you kindly,
Steven
sauravrohilla
859 Posts
1
May 28th, 2013 19:00
Its a SAS pool with 24 disks so I am assuming you must be carving out raid 5 (3+1) on it? I think any multiple of 6 members will give you the best performance...
lets see what others think
regards,
Saurabh
Zikas
278 Posts
2
May 28th, 2013 23:00
Hi Smaeillo,
i am glad to help you and in the past i was in your situation too and there were many people that helped me too.
So keep in mind that everything is circle. You give you take.!!!!
So if it is for ESX hosts and as Saurabh said the SAS Pool contains 24 disks which are very few.
So in my humble opinion try to create big metas with few hypers but always in power of two. I mean two, four, eight, sixteen, thirty two, sixty four.
The larger the storage pool for the allocations is, the greater the number of devices that can be leveraged for VMWare vSphere I/O.
Keep that in mind. Try to create Thin Pools with lots of TDATs disks.
You create meta in order to avoid to reach the Write Pending Device Limit. For one hyper WPDL is 5%.
So you multiply the numbers of hypers the 5% and it is quite difficult to reach that limit.
For ESX try to leep the O.S. in different datastore and in different datastore the data. It is recommended for performance improving.
As i told you for anything you want ask.
Saurabh, Dynamox and many others guys are willing to help you
seancummins
226 Posts
0
May 29th, 2013 06:00
Steven,
In general for Virtual Provisioning (thin / TDEVs) environments I'd recommend 8-way striped metas. The reason for creating metas on top of VP is not to stripe over the physical disks (VP already does this inherently), but to achieve more concurrency on the front-end (and to increase device write pending limits, as NYY mentioned). Every device has its own queue on each front-end port, including meta members. Within a queue, only one IO can be serviced at any given time. If you create an 8-way striped meta, you have eight queues (per front-end path) that can be concurrently servicing IO.
Assuming you're using ESX Round Robin NMP, I would also recommend changing the NMP RR I/O operation limit from the default of 1,000 to 1... this is also to increase concurrency, reduce queue depths, and improve response times.
Thanks,
- Sean
smaiello
61 Posts
0
May 29th, 2013 06:00
WOW This is a lot of information most of which I don't totally understand (yet). So I understand that the hypers allow you to stripe the data across disks (kind of like partitions); however, logically it would seem that you wouldn't get any better performance adding more metas then you have physical disks.
We are currently using RAID 5 7+1. Our environment really isn't I/O intensive. For our current 275 servers we use about 500 IOPs on average.
If I have 24 disks, and create 64 metas how am I actually gaining performance since I'm still spindle bound?
Thanks you kindly,
Steven
smaiello
61 Posts
0
May 29th, 2013 09:00
Sean,
Thanks for the explanation that makes some sense to me (I think). So if we're using 8 front end ports we would really want the metas to be in a quantity of 8 to balance them over the front end ports?
Also yes we were going to use the RR NMP option; however, I don't know where you would set the operation limit. I looked through both the section on the data store and the software initiator. Why would this increase concurrency, and reduce queue depth?
Scratch the finding how to do it I found a blog article:
http://henriwithani.wordpress.com/2012/02/16/changing-round-robin-io-operation-limit-on-esxi-5/
but I still don't understand why it's an improvement.
Thank again,
Steven
smaiello
61 Posts
0
May 29th, 2013 09:00
Ok this makes total sense. Great explanation.
Thanks,
Steven
Zikas
278 Posts
0
May 29th, 2013 10:00
Hi Smaiello,
I believe that Sean explained everything perfect.
And in case that you cannot understand everything at this moment!!!, believe me when you become more familiar with EMC products and VMware products, in a short period of time you will know most if them.
Another think that I would like to mentioned regarding striping is, if you use SRDF and you have an 8-way-meta device, 8 I/Os will be served at once on the other side.
In my humble opinion try to use striped metas instead of concat.