Unsolved
This post is more than 5 years old
6 Posts
0
18436
Stacked 6224's - Optimal configuration of vSwitch for VM Network?
There is a lot of information here so I will first start off with my question, followed by detailed support documentation.
Is my vSwitch configuration for VM and Management traffic optimal? For example, I am wondering why within VMware the MTU is set to 1500 while the corresponding ports on the switch are configured at 9216.
Within vSphere 5.1 I have a standard vSwitch configured for Virtual Machine and Management traffic using NIC0 and NIC1. The vSwitch is configured as follows:
vSwitch
* 120 Ports
* MTU - 1500
* Load Balancing - Port ID
* Active Adapters - vmnic0, vmnic1
VM Network
* Active Adapters - vmnic0, vmnic1
Management Network
* MTU - 1500
* Load Balancing - Port ID
* Network Failover Detection - Link status only
* Notify Switches - Yes
* Failback - Yes
* Active Adapters - vmnic0, vmnic1
At the bottom of my post if the current running configuration.
I have two 6224's stacked. Here's a summary of the configuration (which was configured entirely by Dell):
VLAN's
Default - Ports 1/g1-g5 and 2/g1-g5
20 - Ports 1/g6-g8 and 2/g6-g8
10 - Ports 1/g9-g20, 1/g22-g24, and 2/g9-g24
1/xg1 connects via fiber over to our core set of switches which is why 1/g21 is not part of VLAN 10.
VLAN 20 is used for vMotion.
VLAN 10 is used for iSCSI traffic (EqualLogic PS4100X).
ESX Hosts
I have two Dell PE R910's, each with six physical NIC ports. Two ports are used for iSCSI, two for vMotion, and two for the management network and VM traffic.
The first host is connected to the switch stack as follows:
NIC0 - 1/g2 (VM and Management)
NIC1 - 2/g2 (VM and Management)
NIC2 - 1/g7 (vMotion)
NIC3 - 2/g7 (vMotion)
NIC4 - 1/g12 (iSCSI)
NIC5 - 2/g12 (iSCSI)
The first host is connected to the switch stack as follows:
NIC0 - 1/g3 (VM and Management)
NIC1 - 2/g3 (VM and Management)
NIC2 - 1/g6 (vMotion)
NIC3 - 2/g6 (vMotion)
NIC4 - 1/g11 (iSCSI)
NIC5 - 2/g11 (iSCSI)
The PS4100X (redundant controllers) is connected to the switch stack as follows:
Controller 0
Management - 1/g1
Eth 0 - 1/g9
Eth 1 - 2/g10
Controller 1
Management - 2/g1
Eth 0 - 2/g9
Eth 1 - 1/g10
iSCSISW#show running-config
!Current Configuration:
!System Description "PowerConnect 6224, 3.3.1.10, VxWorks 6.5"
!System Software Version 3.3.1.10
!Cut-through mode is configured as disabled
!
configure
vlan database
vlan 10,20
exit
hostname "iSCSISW"
stack
member 1 1
member 2 1
exit
ip address 192.168.1.31 255.255.255.0
ip default-gateway 192.168.1.1
ip https server
no ip http server
interface vlan 10
name "iSCSI"
exit
interface vlan 20
name "vMotion"
exit
username "admin" password ff22795a8a1287d4d8c945ae8c6a51c1 level 15 encrypted
ip ssh server
!
interface ethernet 1/g1
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 1/g2
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 1/g3
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 1/g4
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 1/g5
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 1/g6
spanning-tree portfast
mtu 9216
switchport access vlan 20
exit
!
interface ethernet 1/g7
spanning-tree portfast
mtu 9216
switchport access vlan 20
exit
!
interface ethernet 1/g8
spanning-tree portfast
mtu 9216
switchport access vlan 20
exit
!
interface ethernet 1/g9
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g10
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g11
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g12
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g13
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g14
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g15
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g16
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g17
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g18
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g19
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g20
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g21
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 1/g22
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g23
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/g24
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 1/xg1
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 1/xg2
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 1/xg3
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 1/xg4
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 2/g1
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 2/g2
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 2/g3
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 2/g4
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 2/g5
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 2/g6
spanning-tree portfast
mtu 9216
switchport access vlan 20
exit
!
interface ethernet 2/g7
spanning-tree portfast
mtu 9216
switchport access vlan 20
exit
!
interface ethernet 2/g8
spanning-tree portfast
mtu 9216
switchport access vlan 20
exit
!
interface ethernet 2/g9
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g10
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g11
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g12
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g13
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g14
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g15
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g16
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g17
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g18
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g19
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g20
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g21
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g22
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g23
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/g24
spanning-tree portfast
mtu 9216
switchport access vlan 10
exit
!
interface ethernet 2/xg1
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 2/xg2
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 2/xg3
spanning-tree portfast
mtu 9216
exit
!
interface ethernet 2/xg4
spanning-tree portfast
mtu 9216
exit
exit
DELL-Willy M
802 Posts
1
January 29th, 2013 09:00
Here is a great article from VMware about enabling jumbo frames (MTU 9200).
kb.vmware.com/.../search.do
It mentions: Jumbo Frames are limited to data networking only (virtual machines and the vMotion network).
You will not want or need Jumbo frames enabled on all connected ports of the switch.
Also here is a good read for Performance Best Practices for VMware vSphere 5.1
www.vmware.com/.../10329
Hope this helps
swoods79
6 Posts
0
January 29th, 2013 11:00
Thank you for the information. It seems that for vMotion and iSCSI jumbo frames is fine but that I should not use jumbo frames on my VM network ports (1/g1-g5 and 2/g1-g5).
I would now like to create LAG's for the ports associated with each vSphere host. Would the following commands be accurate?
console#configure
console(config)#interface port-channel 1
console(config-if-ch1)#description esx1_lag
console(config-if-ch1)#exit
console(config)#interface ethernet 1/g2
console(config-if-1/g2)#channel-group 1 mode on
console(config-if-1/g2)#exit
console(config)#interface ethernet 2/g2
console(config-if-2/g2)#channel-group 1 mode on
console(config-if-2/g2)#exit
console(config)#exit
console#configure
console(config)#interface port-channel 2
console(config-if-ch2)#description esx2_lag
console(config-if-ch2)#exit
console(config)#interface ethernet 1/g3
console(config-if-1/g3)#channel-group 2 mode on
console(config-if-1/g3)#exit
console(config)#interface ethernet 2/g3
console(config-if-2/g3)#channel-group 2 mode on
console(config-if-2/g3)#exit
console(config)#exit
I believe I would then need to edit my vSwitch to "Route based on ip hash".
DELL-Willy M
802 Posts
1
January 29th, 2013 11:00
The configuration for the LAGs looks good. Typically, you would create the port-channel first with the channel group mode command. I'm not sure that it really matters. One thing to remember, once you have configured a physical port to a port-channel then you would make any configurations on the port channel and not the physical port. ie switchport mode, port fast, MTU etc. The physical port would only have the settings for the channel group.
sketchy00
203 Posts
0
January 29th, 2013 12:00
I would first advise that you approach your vSwitch and port group configuration around reference architectures reflecting the number of NIC ports you have on each host. Then build out your vSwitch and port groups accordingly, per best practices for teaming policies, active/standby/unused settings, etc.
You might also want to ask yourself why you want to great the LAG. What does that get you? What does that NOT get you?. If the answers line up with your design objectives, then great. If not, take a step back and look at the loads of great resources out there. (Hint. it may not be the best route for you to go). Lots of great posts on reference arechitectures for vSphere with varous NIC port counts per host (e.g. 4, 6, 8, 10, etc.)