NX4, iSCSI, multipathing, VMware - sequential throughput

Have you checked http://virtualgeek.typepad.com/virtual_geek/2009/01/a-multivendor-post-to-help-our-mutual-iscsi-customers-using-vmware.html

my *guess* is that you need to use more than one LUN to get over 100 MB/s

I would suggest to talk to your local EMC technical contact - for the solutions on Powerlink there are also internal validation reports that usually contain performance information

I did try a quick setup and test with multiple LUNs but I was not initially successful on acheiving higher transfer rates.  I might have to revisit and devote a little more time to that possibility again.

I will have to contact my local EMC technical contact.  I was just hoping that maybe someone might reply with "yep, I did it when I was implementing SQL and this is how...".

Thanks!

Peter, yes I saw that document and I'll raise you with this one:

http://virtualgeek.typepad.com/virtual_geek/2009/09/a-multivendor-post-on-using-iscsi-with-vmware-vsphere.html

I've devoted a lot of time and effort into reading and trying what I've read.  I keep reading about theoreticals and why it should or should not work.  I'm initially asking if someone has done it and can prove throughput of >150MB/s is acheivable (again, throughput of a single sequential read).  Then I'll follow up with "how?".

Are you referring to iSCSI VMFS datastores with VMDKs for the SQL drives sitting on the datastores? Or are you talking about using the iSCSI initator inside of a Windows VM to address the LUNs on the array? The former has a lot of factors that affect IO performance (guest OS disk alignment, potential to use pvscsi (when IOPS>2000), the speed of the nics assigned to the iSCSI port group, mixing IO on a datastore, etc.). The latter does too, though in particular, unless you've dedicated a physical nic to the VM (a bad idea), you're going to get substantially less than 1 Gbps back to the iSCSI target since the bandwidth is shared across multiple VMs.

Which VMware version are we talking about?

In general, I am talking about installing an SQL Windows (Server 2008 x64) machine inside VMware (ESXi 4) and getting over 100MBytes/s sequential read throughput over iSCSI (1Gbps links) to a single target (what appears as a single drive letter on the SQL Windows machine - but it actually multiple drives on the SAN).

***UPDATE!!  YES it can be done...

I was able to accomplish this over the weekend.

- 3 nics on the SAN connected to a gigabit switch - this ports were in their own VLAN.

- 3 nics on the VMware server on the same switch as above, and only in the VLAN mentioned above.  I added a VMware networking vSwitch for the SAN.  I attached a Virtual Machine port group to each nic (only one per nic).  In other words, "SAN nic0" was vmnic0, "SAN nic1" was vmnic1, and "SAN nic2" was vmnic2.  When setting up the SQL VM in VMware ("Edit virtual machine settings"), add those 3 network adapters in addition to the default VM Network adapter.  Basically, you are passing through the adapters to the Windows VM.

- After installing Windows (server 2008 x64), set each of those nics to have their own individual IP addresses (same subnet as the SAN).  Then, setup the iSCSI initiator to connect to the SAN.  In your iSCSI connections properties, setup MCS (Multiple Connected Sesstions).  There will be one already there addressed by "0.0.0.0/43860" or something like that.  That will actually be one of the adapters.  You have to add the other 2 "SAN" adapters.  I added one and watched the Network Utilization in Task manager to see if one picked up from 0%.  I then did the same to get the remaining one.  I verified the MCS policy was Round Robin.  Done.

Now, with I/O read requests (heavy ones), I was able to get 150MB/s and sometimes peak over 200MB/s.  What made the last bit of difference was presenting multiple LUNs (2 of them to be exact) to the Windows VM for the SQL databases.  I setup one filesystem on the SAN, and chunked it up into 2 LUNs.  Then in the Windows VM, I saw 2 disks and created a STRIPED volume (64k allocation unit size).  I was able to get sustained 190MBytes/s throughput (peaks up into 225-230MB/s).  3 LUNs (3 Windows disks Striped) - about 180MB/s sustained.  Last, a single LUN presented as a single disk in Windows - about 120-150MB/s.  Not sure why 2 LUNs were faster than 3 LUNs, but I'm guessing it has to do with overhead at the backend at the SAN.  If someone knows why, I'll listen.

I also tried 2 nics on the SQL VM.  3 was better than 2 as expected, but not a lot.  When a test of 3 nics provided 190MB/s, the same test at 2 nics produced 140MB/s.  Each physical nic did work harder when there were 2 of them, versus 3.

Another test was 3 nics and 2 LUNs, but not in a Windows STRIPED volume.  It dropped to about 140MB/s.

So, in conclusion, the best scenario I have tested so far was 3 iSCSI (1Gb/s) nics and 2 LUNs STRIPED together (64k alloc unit) in Windows.  If someone finds faster, I'm all ears.

Does this guarantee superior disk subsystem performance in SQL?  Of course not, we need to consider write throughput, other systems sharing the SAN, etc.  But, this is a good start.     Or, is this (running Windows striped volumes over iSCSI) totally unsupported and a recipe for disaster?? 

Time will tell, that's why we make backups...