Disk Queued I/O

I would suggest that you visit the HBA vendors web page, go to the EMC section for the drivers and check that you have the EMC recommended drivers and Microsoft hot fixes installed.

If that does not resolve your issue then you should probably open a case with EMC to check out the configuration on the cluster. When you open a case be sure to specify that this is a host issue - you will need to provide what's called an "emcreport" or host grab. You can find a program on PowerLink for all the operating systems under "Host Utilities" - find the one for Windows called "emcreports". There will also be a README for running this. Once you get the tool, run the grab and attach to the case. You should also get the spcollects for both SP's also.

Home > Support > Product and Diagnostic Tools > Grab Utilities

glen

How much queued IO do you have? Even on my larger systems that use DMX storage I see queued IO. I little queuing is ok, but if you are queuing more than your queue depth you will have serious performance issues. Also, how many active paths to the array do you have?

are those hosts go thorugh ISL or core connections?

Disk concurrency is good as long as you dont get queue depth full errros obviously you are having issue with performance but its strange that the Cluster keeps failing over due to disk bandwidth issue. Microsoft have informed you that its a backend issue so i would presume that you are pushing large i/o out to the disk. OLTP database normal behaviour is to produce small random i/o so the bottleneck would tend to be on the frontend.

what amount of bandwidth are you producing. What speed is the backend 2 / 4GB and what type of DB is it OLAP or OLTP?

My recommendation would be to collect perfmon disk metrics data at the host level, to see what the host IO Profile looks like from bandwidth, iops, service time and queued IO's perspective. Then move to the array performance analysis.