Unsolved
This post is more than 5 years old
4 Posts
0
26960
January 31st, 2013 03:00
R510 SAS RAID 10 Hosting Hyper-V poor disk speed
Setup:
R510 PERC H700: 8 virtual disks in total. System drive\OS on RAID1, other 7 VD on 4 SAS 600GB as RAID10
Read Policy: Adaptive Read Ahead
Write Policy: Write Back
Disk Cache Pol: Disabled
OS: W2K8 R2 ENT, Hyper-V + failover clustering
Each VM hosted in separate volume on RAID 10, SIOS DataKeeper Cluster Edition replicating the 7 volumes to two identical R510's
VM's: One Exchange 2010, one SQL2005, Citrix Server, File and Print (Os mixes of W2K8 and W2K3 64 Bit)
Problem:
I am seeing very high average disk queue lengths in all the 7 VM's (maxing at 100 in performance monitor) whenever I try and do large(ish) file transfers (2Gb plus). Other symptoms: Exchange VM slows right down if anyone emails attachments of +7Mb. File and Print Server, long time to enumerate folder structures etc... I've tried updating BIOS, RAID f/ware Broadcom network adapters (against my better judgement I've got a Broadcom LACP team upto my switch for VM LAN access). I'm also seeing pretty sluggish throughput with IPERF from other LAN computers into the Hyper-V (R510's).
Kind of running out of ideas here, any help appreciated


Daniel My
12 Elder
•
6.2K Posts
0
January 31st, 2013 09:00
Hello gordyr
Is this a typo or do you have 7 RAID 10 virtual disks sliced across 4 HDDs?
How many users are there?
In your testing is this during production hours or off peak hours when no one is using the server?
Could you explain in more detail how you have this set up? Is the cluster traffic on the same VLAN as the regular traffic? Is it configured as a LAG or did you use the Broadcom Advanced Control Suite to create a team? Do you have one NIC that all of this traffic is going across?
Thanks
GordyR
4 Posts
0
February 1st, 2013 00:00
Hi Daniel
7 VD= 7 virtual disks, spanned across 4 physical disks configured as a RAID10 (each VD is a Windows volume that hosts it's own VM)
^ I'm guessing this is where my bottleneck is in terms of disk I/O
How many users? 15 tops
Broadcom BACS4 configured LACP team, upto an Extreme XOS switch with ports configured as a L3 LACP port team.
The server itself has 4 nics (2 broadcom, 2 intel) the intel nics are configured on a different subnet for exclusive use by the underlying volume replication app "SIOS DK Cluster", the broadcom team I am using for cluster vm migration and LAN access
Daniel My
12 Elder
•
6.2K Posts
0
February 1st, 2013 09:00
Okay, you have 1 RAID 10 virtual disk with 7 windows volumes/partitions. That is fine.
I think this is your problem. You have a Link Aggregation Group/Port-Channel/Channel-Group created on the switch, and you have a team created in BACS. You should configure one or the other. If you have a BACS team and a LAG then they will conflict and cause a lot of packets to be improperly routed and resent.
LAGs and teams function differently. A LAG will not spread a data stream across multiple ports, but a team will. If exchange is pushing through a 7MB attachment then the LAG sees that as a single data stream and will only use one port. If you have teaming configured as well then teaming will attempt to spread that traffic across multiple ports. The switch will not accept the packets sent on the other ports by the team, and they will be resent over and over until both the LAG and team pass the data in a method both can agree on.
The other concern I have is with multicast traffic from the cluster. Even though the cluster traffic is segregated on a different subnet it may still multicast across the switch. Is the cluster traffic on its own VLAN?
Aside from the team in BACS I think your issues lie with traffic segregation on the switch. If it is a Dell switch then I would suggest creating a post in the networking forums. If it is a 3rd party switch then I would suggest contacting the vendor on how to configure for multicast traffic. I know that some of our switches are not able to properly handle multicast traffic.
Thanks
pcmeiners
6 Operator
•
1.8K Posts
0
February 2nd, 2013 15:00
TO add...
Teaming will not increase throughput from a single client, only helps with multiple accesses from multiple clients. On the NIC interface unless your using IP6, disable it. also with your low number of WKS, your likely to be better off with disabling flow control in the NIC properties. Have you run chkdsk? Turn off SMB signing.
Have you read this? tweak carefully....
http://msdn.microsoft.com/en-us/library/windows/hardware/gg463392.aspx
GordyR
4 Posts
0
February 6th, 2013 01:00
Thanks for the suggestions Daniel, I'll look at my BACs config. I have had some feedback on the MSDN forum re: 4 spindle RAID 10 with 7 VD not being an efficient config for Hyper-V, I may take a look at testing the VM on a RAID1 to see if I can improve disk I/O
Daniel My
12 Elder
•
6.2K Posts
0
February 6th, 2013 09:00
I think there is some confusion regarding terminology. Yes, 7 Virtual Disks/Sliced arrays is bad for performance. Based on what you described I don't think you actually have 7 Virtual Disks/Slices. I think you have 7 partitions within windows on a single RAID 10 virtual disk. There is a huge difference between 7 virtual disks and 7 partitions. Virtual disks are created on the RAID controller. Partitions are created inside the operating system.
Thanks
GordyR
4 Posts
0
February 7th, 2013 00:00
no it's 7 VD's configured at the RAID utility, not 7 partitions configured within Win