Slow SAN performance with high latency

Question

I started a new job a month ago and identified some health problems with the current infrastructure. One is the PS100 SANs at the main and dr sites.

To troubleshoot I directly attached a win2k8r2 physical server with a qlogic iscsi hba card through a crossover cable to one of the members of the SAN and ran some IOmeter tests.

Sequential read, 0% random, 32k, with one work process and a 4gb test file yields 970iops and 32MB/s with 1-248ms latency. Bumping that up to 8 worker processes yields 2900iops/95MBs/2-285ms.

The array is configured with one large (default) storage pool across three members and all disks, with raid 10 on each member. I do see that one of the members has 750gb disks, while the other two members have 500gb disks, and the third member disks have slightly longer disk latency times because of their size.

Where do you think I should start to look on how to improve performance?

lcasteel · Answer

Edit: Meant Raid50 on each member.

lcasteel · Answer

Directly attaching the iSCSI was a first step measure to rule out major switch problems. Setting up MPIO after I hit the limit of the 1 GB HBA was my next step.

I mixed up my terminology, I meant outstanding IOs were bumped up to 8, not workers.

I have been watching SANHQ, which is why I do think there is a problem: http://i.imgur.com/2q62i.png. This is on a production SAN. As you can see, as IOPS rarely go over 2000, latency is often high, and I've never seen a transfer rate over 100 MB/s.

Thanks for the info, this gives me some ideas.

EqualLogic

Slow SAN performance with high latency

Was this post helpful?