Unsolved
This post is more than 5 years old
46 Posts
0
805
Queue depth: DMX4 & ESXi
Trying to determine the correct queue depth for an ESXi cluster with striped TDEV meta's.
The general rule of thumb I've used is ( 8 * <# meta members> ) / <# paths to lun>
15x hosts in the ESXi cluster, 2x HBA's each (2x paths per host), each lun is a 16x striped TDEV. Each FA have 2x ESXi clusters (30x hosts per FA port). So 8 * 16 / 30 = 4.2 queue depth.
Because the ESX guys did some things and all the ESXi servers are now running at the defaults (32) that I didn't notice before. The servers don't appear to have any notices of queue fulls in the ESX logs which leads me to think the hosts are not receiving queue full's on the DMX, but in SPA it looks like I've got huge queues on the FA's (the description for Queue Depth Range metric is horrible so hard to tell). I have a pretty constant peak of 200 of AVG Queue Depth Range 7 and have spikes into the range of 8 & 9 (I'm assuming higher numbers are bad and I'm receiving lots of bursty traffic).
Nobody is yelling or screaming at us but it seems like a depth of 4 should be put back into place OR has anybody tried the adaptive queueing on ESXi (vmware KB 1008113)? Doing a search for the configurables on powerlink (QFullSampleSize / QFullThreshold) returns nothing. If we were to use adaptive queueing any idea as to what the values should be? Having everything set manually is nice and predictable but it may limit throughput that adaptive might allow otherwise unavailable capacity to be used since the host won't queue up more io's.
Booyah2
184 Posts
0
May 16th, 2011 07:00
Well, I'm not an expert on the topic, but I would recommend leaving it at the EMC recommended value unless you had an issue that was determined by a performance specialist that required the Queue depth to be changed. Having a high Queue depth is not necesarily an issue...
I had a performance engineer describe it to me this way:
If you had 1400 logical volumes mapped to a FA and each one had 1 I/O queued, you would see 1400 in the queue and seeing that the FA can work on each logical volume individually it would not be a big issue. Now if you had the same 1400 logical volumes and there were 7 of them with 200 I/O’s queued, you would still have 1400 in the queue, but on 7 volumes to work on and each one would have a queue of 200. If each I/O took 1 ms, then that last I/O in the queue would have a response time of 200ms.
Hope this helps...
Thom
Forum Reminder:
If you find any of the replies to your question as helpful, please be so kind as to mark replies as being "helpful" or "answered". This helps us track our success with the Forum.