MD3200i iSCSI TEAM NIC (802.3AD)

You never team NICs when using iSCSI; each NIC/port needs it's own IP address.

To get an idea about how to IP the SAN's iSCSI ports, check the default settings:

Controller 0 iSCSI port 0: 192.168.130.101

Controller 0 iSCSI port 1: 192.168.131.101

Controller 0 iSCSI port 2: 192.168.132.101

Controller 0 iSCSI port 3: 192.168.133.101

Controller 1 iSCSI port 0: 192.168.130.102

Controller 1 iSCSI port 1: 192.168.131.102

Controller 1 iSCSI port 2: 192.168.132.102

Controller 1 iSCSI port 3: 192.168.133.102

When you're using 2 NICs in the server for iSCSI, give them something like 192.168.130.50 and the other 192.168.131.50. The server can then make 4 iSCSI sessions and if the MPIO software is installed it can loadbalance and do failover between those sessions.

So with the network as described above the controller network ports on the .132.xxx and .133.xxx subnets are never utilized by the servers since there is no network interfaces configured for the .132.xxx or .133.xxx on the servers ?

Thank you for your time.

-mike

This is correct. If you're doing 2 servers, the 2nd could use the other set of ports (the 132 and 133 subnets), or if you opt to use 4 NICs for iSCSI on 1 server, each NIC uses one of the 4 subnets.

yeah..  It gets tough doing 4 nics and the switching to properly support it.   I'd expect that 4 nic per host setup would be pretty impressive performance wise, because two is already way decent from my experiences.

So how would I setup 3 Servers with 6 NICs (2 teamed for Management / LAN)?

That depends on what you want to do and how many (dedicated iSCSI) switches you have.

Hardware:

3x Dell R710 Servers with 6 NICs (ESXi Servers)

1x Dell R310 Server with 6 NICs (vCenter & Management)

2x Dell PowerConnect 6224 Stacking Switches

1x Dell MD3200i SAN

1x Dell MD1200 DAS

I need to setup redundancy and high availability.  I have NIC 1&2 teamed on each ESXi Server for the Management Network.

I have ESXi installed on the R710s, vCenter installed on the R310 and a Cluster setup.  I have the SAN setup with 3 logical disks for OS, Database and Storage.

I need guidance in setting up the ESXi networking, iSCSI, LUNs, VLANs and making everything work together.

Are there any guides for all of this?

So I guess no one gave any thoughts to this? I'm in a similar boat, I have 3 esx hosts (10 PNICS each) and a dual controller 3220i.

On the hosts that is.

So I still have 6 NICs left over, any recommendations on how to best divide them up?

Devote 4 nics each on a different subnet to the iSCSI.     You should have 4 dedicated switches to accomadate this. Or at least 4 Vlans dedicated to this.

I'd just leave the ports on the san at default,  but you want 4 different subnets.  One port on each controller should have the same subnet. This should yeild 8 valid paths,  4 active, 4 standby, from each host.

Found this guide:

www.dell.com/.../md-family-deployment-guide.pdf

Definitely don't use that document if you want a decent setup. I'm not sure what that writer of that document was thinking, but they seem to have taken the Equallogic documentation and copied and pasted it and assumed it was fine for an MD-series SAN.

You're better off using this article: www.delltechcenter.com/.../VMware+ESX+4.0+and+PowerVault+MD3000i.

Just do the same thing, but then with 4 subnets. With vSphere 5 you can do all the steps in the GUI (set frame size and bind the vmkernels to the iSCSI initiator, which may make life a little easier. Some of the CLI commands are also a bit different, so if you want to script the iSCSI config, you'll have to look around for how to do some of the steps with the new commandline.

So I have things setup, but I don't know if the redundancy / high availibility is setup correctly...

Dell had me setup the SAN like this:

Controller 0 iSCSI port 0: 192.168.130.101

Controller 0 iSCSI port 1: 192.168.131.101

Controller 0 iSCSI port 2: 192.168.132.101

Controller 0 iSCSI port 3: 192.168.133.101

Controller 1 iSCSI port 0: 192.168.130.102

Controller 1 iSCSI port 1: 192.168.131.102

Controller 1 iSCSI port 2: 192.168.132.102

Controller 1 iSCSI port 3: 192.168.133.102

So I know I need to create 4 VMKernal Ports on each of the 3 ESXi Servers:

ESXi01:

192.168.130.10

192.168.131.10

192.168.132.10

192.168.133.10

(and then .11 on ESXi02, .12 on ESXi03)

My question is - can all the VMKernal Ports be part of the same vSwitch on each Server with all 4 NICs added, or do they need to be seperate vSwitches and attach to each NIC?

Also, for Jumbo frames - do I need to enable higher MTU on the vSwitch or each individual vmKernal iSCSI port in the vSwitch?

I'd start out without Jumbo frames.  Most environments I see?  It actually decreases IOPS some.

I see no reason to devote more than 4 nics per host to MD dual 4 port controller sans with 1gb.