Number of I/O Queue and Queue Depth for CX3-20 ?

Per fibre port on each SP you can cumulative add up all queue depth's up to a maximum of (rule of thumb) 1600.

Let me rephrase that: if you add up all the queue depth's (execution throttle's), you should not exceed 1600 per fibre port on each SP. So keep that in mind when zoning hosts to ports.

Dear Rob,

Many thanks for your advice.

Can I say that the storage array (SP) will allocate one I/O queue for each LUN created (No matter in which RAID Group) ?

Is there any typical LUN queue length ? Is it determined by the Storage Array or can we configure it manually ?

Thanks

Message was edited by:
TonyJK

As far as I know there's no specific lun queue length, similar to the Symmetrix's write pending ceiling, but 1 I/O queue per lun is correct, yes. The host will cause this queue length, not the Clariion as outstanding I/O's come from I/Os pushed into the Clariion by hosts.
You cannot configure this queue yourself. In fact, I doubt that it is configurable at all.

Be aware not to cross the maximum queue depth (called "execution throttle" in Qlogic HBAs) per storage processor port. This max is almost 2000, but rule of thumb (best practice) is not to cross 1600.

Rob & Tony,

Let me try to explain the queue depth a little further¿

As already correctly pointed out CLARiiON has an overall queue depth limit. This limit is shared among all connected hosts.

The Host target queue depths: sum of all host target queue depths < 1600 per SP port

Maximum LUN queue depth: setting for a single LUN < (14 x #disks) + 31

Now if your typical HBA has queue depth of 32 it will never exceed the LUN queue depth. But if you have multiple paths to the same LUN then that¿s when it gets more existing¿

Please also see the excellent EMC Clariion Best Practices for Fibre Channel Storage FLARE xx Release Guide for some more information and a better explanation of the above.

I hope this helps
Calle

With #disks you mean the physical spindels that take part in a specific lun ?

So if a lun sits on 5 physical spindels, the lun queue depth would be 14 x 5 + 31 = 101 ? Even if the lun is the only lun on these disks and even if the lun takes up all space on those disks (like you should design your ATA usage) ?

No not physical spindles but data spindles in a LUN. And my apologies it is +32 and not +31!
So, in a 4+1 R5 it would be 4*14+312 = 88 or in 1+1 R10 it would be 1*14+32= 56¿

So if I have 20 small (or big) luns, I would have to do the math 20 times and sum it all up and set that in the HBA (with SANsurfer for example or through the HBA's BIOS settings) ?

Or should I simply set the execution throttle to the minimum of all luns I just calculated ? Just to make sure I won't go over the queue depth of any specific lun...

The latter please as otherwise you will for sure exceed the 1600 (+extra allocated as a buffer) limit per SP port¿

Using Emulex as an example, if you download the ¿EMC Fibre Channel with Emulex Host Bus Adapters in the Windows Environment¿ you will see that the for Clariion connections the MaximumQueueDepth will be set to 32. In most cases this is a fair number to start off with¿

But as previously mentioned the:

Host target queue depths are the sum of all host target queue depths < 1600 per SP port
The maximum LUN queue depth setting for a single LUN < (14 x #disks) + 32

Now if the CLARiiON per-port or per-LUN max queue depth is exceeded by a request from one if its attached hosts, the array will return a SCSI QUEUE FULL status. This may result in a temporary condition of decreased host thread parallelism and thus noticeable reduction in total array throughput. Gracious recovery from this condition is left largely to the host (and driver) in question. Furthermore, there is absolutely no guarantee that the host generating the most load will be the recipient of the QUEUE FULL condition

So the maximum LUN queue depth limit is calculated as:
(#threads per HBA) = 1600 / (#HBAs connected to this-port)
LUN queue depth = (#threads per HBA) / (#LUNs on this port for this host)

Windows has no overall limit for the depth of the queue at the target ¿ it can only be adjusted by manipulating the LUN queue depth. The default value of 32 may be too high in certain environments. For example 100 LUNs being accessed by a single host through one SP port will exceed the 1600 limit¿

The SCSI QFULL ¿condition¿ can be best be described as a momentarily and random freeze of IO. If you think you are effected you could use Navisphere Analyzer (Rel 24 or later) to verify the port-queue full counter.

I hope this helps?

I'm curious, what is your relationship with Clariion (how is it possible that you know so much about the subject) ? Everything you said indeed sounds familiar (I did the Clariion Performance Training almost 2 years ago)


I'm glad I work mainly with Symmetrixes as those have no such queue depth limit

Similar, I try to read a lot of documents and been on the expert level courses. Talking of which, they are brilliant especially if you mange to get Steve Stead = Clariion guru as your instructor. There you go that¿s my EMC advertisement for the day done

I work mainly with the CLARiiONs but would also like to get to know the Symm. But I guess my first love will always be the CLARiiON

Steve Stead... yeah: he was my instructor too !!!! He has Clariions at home ! And he lives in an appartment building as far as I know...... The neighbors love him, I guess....

My first love is Clariion as well, but working with the Symm now for almost 2 years, I fell in love with those monsters..... Can you blame me ?