This post is more than 5 years old
8 Posts
0
79541
MD3220i Setup with ESXi - IP Addressing & Subnetting
Hi Everyone
I'm setting up an MD3220i with two switches for redundancy and one ESXi server. I've followed all the documentation but would like to run it by everyone as I'm not sure I've done it right.
I've enclosed a diagram of exactly how my setup looks (except for my ESXi server only has two port, so one line into each switch) and placed my IP configurations for the PowerVault at the bottom of the post.
ESXi:
I've created a switch with virtual kernels for each physical port and have bound them to each physical port, configuration below
iSCSI01
vmk1: 10.0.1.3
iSCSI02
vmk1: 10.0.1.4
I've carved up LUNs on the PowerVault.
Now my questions are as follows:
- Are the IP address for the PowerVault correct? I have subnetted ports from each card in groups, and just specified a random gateway.
- Does the gateway matter?
- Does the IP address of the vmk's matter? I've left the gateway for these unspecified, does that matter? What should the gateway be?
- Can anyone explain why I have bound each vmk to a physical port in simple terms, I did this just because it said to in the documentation?
As you can see, my weak points are IP Networking. What Im really looking for are simple explanations, I've read a lot of documentation provided by Dell and to be honest I've just confused myself.
Any information would be great. Thanks everyone.
Below is my setup:
Controller 0:
IN 0 Port:
IPv4:
IP address: 10.0.1.1
Mask: 255.255.255.0
Gateway: 10.0.0.10
IN 1 Port:
IPv4:
IP address: 10.0.2.1
Mask: 255.255.255.0
Gateway: 10.0.0.10
IN 2 Port:
IPv4:
IP address: 10.0.3.1
Mask: 255.255.255.0
Gateway: 10.0.0.10
IN 3 Port:
IPv4:
IP address: 10.0.4.1
Mask: 255.255.255.0
Controller 1:
Management Port:
IPv4:
IP address: 192.168.69.36
Mask: 255.255.255.0
Gateway: 192.168.69.37
IN 0 Port:
IPv4:
IP address: 10.0.1.2
Mask: 255.255.255.0
Gateway: 10.0.0.10
IN 1 Port:
IPv4:
IP address: 10.0.2.2
Mask: 255.255.255.0
Gateway: 10.0.0.10
IN 2 Port:
IPv4:
IP address: 10.0.3.2
Mask: 255.255.255.0
Gateway: 10.0.0.10
IN 3 Port:
IPv4:
IP address: 10.0.4.2
Mask: 255.255.255.0
Gateway: 10.0.0.10
Dev Mgr
4 Operator
4 Operator
•
9.3K Posts
1
March 30th, 2011 11:00
You will want to read this post on Dell's 'delltechcenter' website. It's a very useful document: http://www.delltechcenter.com/page/VMware+ESX+4.0+and+PowerVault+MD3000i.
I recommend that you split your iSCSI into 2 different vSwitches with 1 NIC each. If you are absolutely determined about using a single vSwitch with 2 VMKernels on 2 different subnets with 2 NICs, you need to isolate your VMkernels to each only use 1 NIC. You can do this in the vSphere client by going to the vSwitch and then the VMkernel properties. On here you go to the NIC teaming tab, override the defaults. On iSCSI01 move vmnic2 down to "unused", and on iSCSI02 move vmnic1 down to unused.
If you then pull another "esxcfg-vswitch -l", it'll look something like this:
Other than that, I'd suggest to check if your iSCSI switches support jumbo frames, flow control, portfast (spanning-tree option) and maybe even multicast/broadcast storm control (leave unicast storm control disabled though (if you cannot individually set these 3, leave all 3 disabled)). If your switch can be optimized for iSCSI, I'd check that link and use that to complete recreate your iSCSI setup (from the command line) to add support for jumbo frames for the iSCSI traffic (and maybe even for vMotion if you have that licensed).
expsup
8 Posts
0
March 30th, 2011 05:00
Anyone out there?
Dev Mgr
4 Operator
4 Operator
•
9.3K Posts
0
March 30th, 2011 07:00
The IP scheme looks fine.
On the server I wouldn't set a gateway for the iSCSI VMkernels as a server can only really have 1 gateway and it should be the one on your LAN and not the iSCSI network(s). For the SAN the gateway for the iSCSI ports doesn't matter, so they are fine as they are, but leaving them blank is also a valid option.
The IP addresses of the VMK's is important as they need to be on the subnets that the vmnic attached to that VMK has access to.
If you don't 'tell the iSCSI initiator which VMKs it can use, it'll only use 1', which means you get failover between the 4 NICs, but no loadbalancing (difference between using 1 NIC at a time (1 Gbit/s) or all 4 NICs at the same time (4 Gbit/s).
Can you post the output of:
- esxcfg-vswitch -l
- esxcfg-vmknic -l
- esxcli swiscsi nic list -d vmhba## (where ## is the vmhba number for your software iSCSI initiator (usually vmhba32 or higher).
expsup
8 Posts
0
March 30th, 2011 08:00
Hi
Thank you for your reply, I was really bashing my head against a wall thinking I may have configured it wrong.
Please find the outputs below:
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch0 128 5 128 1500 vmnic0,vmnic3
PortGroup Name VLAN ID Used Ports Uplinks
VM Network - Explorer 0 1 vmnic0,vmnic3
Management Network 0 1 vmnic0,vmnic3
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch1 128 5 128 1500 vmnic1,vmnic2
PortGroup Name VLAN ID Used Ports Uplinks
Internal Network 0 0 vmnic1,vmnic2
iSCSI02 0 1 vmnic2,vmnic1
iSCSI01 0 1 vmnic1,vmnic2
I will be adding additional ESXi servers to this environment, each ESXi server will only have two dedicated NIC's for iSCSI traffic and two for internal network.
Thanks
expsup
8 Posts
0
April 1st, 2011 07:00
Hi
Thank you for your reply. I have taken your advice and created two vswitches and configured round robin. Ive also changed the subnets on the PowerVault, so now there are only two subnets. Please see config below (taken out management network info to make post smaller).
Do you think this looks right?
Thank you again for your excellent help.
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch1 128 3 128 1500 vmnic1
PortGroup Name VLAN ID Used Ports Uplinks iSCSI01 0 1 vmnic1
Switch Name Num Ports Used Ports Configured Ports MTU Uplinks
vSwitch2 128 3 128 1500 vmnic2
PortGroup Name VLAN ID Used Ports Uplinks iSCSI02 0 1 vmnic2
--------
Interface Port Group/DVPort IP Family IP Address Netmask Broadcast MAC Address MTU TSO MSS Enabled Type
vmk1 iSCSI01 IPv4 10.0.1.11 255.255.255.0 10.0.1.255 00:50:56:73:01:0e 1500 65535 true STATIC
vmk2 iSCSI02 IPv4 10.0.2.11 255.255.255.0 10.0.2.255 00:50:56:7e:46:34 1500 65535 true STATIC
vmk1
pNic name: vmnic1 ipv4 address: 10.0.1.11 ipv4 net mask: 255.255.255.0 ipv6 addresses: mac address: 00:1e:0b:1e:b6:7c mtu: 1500 toe: false tso: true tcp checksum: false vlan: true vlanId: 0 ports reserved: 63488~65536 link connected: true ethernet speed: 1000 packets received: 36436 packets sent: 6992 NIC driver: bnx2 driver version: 2.0.7d-3vmw firmware version: 1.9.6
vmk2
pNic name: vmnic2 ipv4 address: 10.0.2.11 ipv4 net mask: 255.255.255.0 ipv6 addresses: mac address: 00:10:18:94:04:7c mtu: 1500 toe: false tso: true tcp checksum: false vlan: true vlanId: 0 ports reserved: 63488~65536 link connected: true ethernet speed: 1000 packets received: 12558 packets sent: 3590 NIC driver: bnx2 driver version: 2.0.7d-3vmw firmware version: 5.0.6