Connections to multiple NICs

Question

Hi,

Quick question. We have a PS6000 with all 4 NICs for traffic. We have a VMware host with a 10gb NIC and a 1GB NIC for iSCSI.

At present, we have fixed path to use the 10gb primarily, with the 1gb in place just for redundancy.

I want to make best use of the 10gb pipe and spread the load across all 4 NICs of the PS6000. If we had 4x1gb NICs at the VM end, then the MPIO driver would nicely balance everything across all 4 connections, with a 1-1 mapping of the NICs.

Is anyone aware of a way of getting the 1x 10gb NIC to use all 4 ports / IPs at the EQ end for maximum bandwith?

Thanks

dkelly84 · Answer

Sorry I think my description wasn't clear. We don't have 1x10gb and 1x1gb active - we have 2 vmkernel ports, each bound to one of the adapaters, but we don't have multipathing enabled, we are using VMWare Fixed path mode to ensure that all the data goes down the 10gb NIC. Only in the event of that NIC failing will data use the 1gb link.

What I'm looking to achieve, is have the 1x10gb NIC access all 4 ports of the PS6000 for each datastore, rather than just the 1.

Origin3k · Answer

If you have an active 10GGbE and single 1GbE for redundancy you have a non working swISCSI setup when having a >= vSphere 4 environment. If youre on 3.5 or older this was possible. The MPIO on a vSphere 4 host only works with VMKernel which have only one active Uplink and all other are marked as Unused.

Create multible VMKernen Port and bind these to the swISCSI module to get MPIO working. With only a single physical NIC you dont have redundancy on pNic, Cable or Switch . So you need another 10GbE NIC to get at least 2 Ports.

Always install the EQL MEM VIB, also when having non vSphere Enterprise+ and apply the other bestpractices for FlowControl, Delayed ACK or disabled LRM.

Regards

Joerg

Dev Mgr · Answer

Typically each NIC runs 1 single iSCSI session to a given member (per volume). So if you use a 10Gbit NIC to a 1Gbit port on the SAN, you'd only get a single 1Gbit link.

This wouldn't be supported/recommended, but you could create multiple vmkernels that are bound to the 10Gbit NIC (and 1 more for the 1Gbit NIC), and see if it will make multiple connections from the 10Gbit NIC to the iSCSI ports on the SAN. I have never tried this myself (and wouldn't recommend it), but you can try it.

dkelly84 · Answer

Based on the suggestions here, I've tried adding additional vmkernel ports bound to the same 10gb card - This seems to work fine and seems to solve my problem, but is bit of an ugly solution

However, after reading the latest VM iSCSI deployment guide (here if you're interested: www.vmware.com/.../iSCSI_design_deploy.pdf ), section 2.4 seems quite interesting, so I may throw a vShere 5.5 box together as a test to see how it behaves. I will update with my findings.

dkelly84 · Answer

Thanks Don, I'll take that into consideration. At present it's more of an exercise in squeezing every last drop of performance out of the SAN before replacing it next fiscal year (Most likely with the 10gb version of the same array). We're a University that does a lot of media work, so we're inevitability moving large files about regularly, not to mention backing them up.

If it turns out dropping back to several 1gb up links ia for the best, then so be it.

EqualLogic

Connections to multiple NICs

Was this post helpful?