Unsolved
This post is more than 5 years old
48 Posts
0
4795
jumbo frames and vsphere 5
We've been testing with jumbo frames on our vnxe3100 connected to a vsphere 5 installation.
Setup is pretty simple, vsphere host with vmknic and vswitch set to mtu of 9k, cisco switch set to 9k mtu(vlan and interface) and single interface on vnxe3100 set to 9k.
When testing with an iscsi service on the vnxe, you can see the TCP handshakes en iscsi commando's going back and forth. But when vnxe starts to send a jumbo frame, the vsphere hosts is receiving it with the wrong SEQ number and starts to send dup ack's and after a while it's resetting the TCP connection.
The same sequence of events keeps repeating it self, it looks like we're always missing the first jumbo packet as the difference in seq is around 8890 between the last good packet and first received jumbo packet.
Is there a tcpdump on the vnxe so we can have a look what kind of packets are leaving the vnxe as it looks like we're loosing some jumbo packet somewhere.
Regards,
John Grinwis
jjgrinwis
48 Posts
0
November 11th, 2011 03:00
we added some other iscsi appliance to the network, configured jumbo frames on this appliance and working without any problem to our vsphere 5 host. We're going to open as it looks like vnxe is doing strange things with the jumbo packets
comdivisionys
18 Posts
0
November 15th, 2011 11:00
Could you find out more on this one, as it seems weird, I only turned Jumbo Frames on for a customer, but haven't really fully checked it...
jjgrinwis
48 Posts
0
November 16th, 2011 00:00
we found the problem, it's the combination jumbo and vlan tagging on the vnxe.
If you enable vlan tagging and jumbo's on your interface it's not working, looks like the vnxe is not allocating enough space for the 4 extra bytes that makes up the 802.1Q vlan tag in the ip header.
max packet size when using jumbo and vlan tagging on vnxe:
~ # vmkping -c 5 -d -s 8968 172.16.8.61
PING 172.16.8.61 (172.16.8.61): 8968 data bytes
8976 bytes from 172.16.8.61: icmp_seq=0 ttl=255 time=0.944 ms
8976 bytes from 172.16.8.61: icmp_seq=1 ttl=255 time=0.753 ms
8976 bytes from 172.16.8.61: icmp_seq=2 ttl=255 time=1.023 ms
~ # vmkping -c 5 -d -s 8969 172.16.8.61
PING 172.16.8.61 (172.16.8.61): 8969 data bytes
--- 172.16.8.61 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
max packet size when using jumbo and no vlan tagging on vnxe:
~ # vmkping -c 5 -d -s 8972 172.16.8.61
PING 172.16.8.61 (172.16.8.61): 8972 data bytes
8980 bytes from 172.16.8.61: icmp_seq=0 ttl=255 time=0.857 ms
8980 bytes from 172.16.8.61: icmp_seq=1 ttl=255 time=0.701 ms
Now everything is working fine, jumbo's going back and forth between vnxe and vsphere 5 host:
Port InPkts 1549-9216 OutPkts 1549-9216
Po2 38 207945
In the tcpdump output you can see we're missing one jumbo and looks like that's not being send out by the vnxe.
As the jumbo's entering the switch are delivered to the host but the host is missing a jumbo packet.
EMC engineering is already looking into this.
jjgrinwis
48 Posts
0
November 19th, 2011 07:00
The VNXe will only support jumbo's if not using any tagging. With the extra 802.1Q vlan tag, jumbo packet is getting too big for the VNXe and it's dropping the packet. So only support for ethernet frames with a max length of 9104 bytes, so no support for baby or super jumbo's.
cmalpass
1 Message
0
November 30th, 2011 06:00
I ran into the same issue with a similar setup. We have 2 Dell PE R710s w/ESXi5, 2 Cisco 2960S switches stacked, and the VNXe 3100. We have user accessible CIFS traffic on VLAN 1 and ESX only NFS storage traffic on VLAN 2, where we need the jumbo frames. We have both VNXe ports etherchanneled and configured as trunks, all other ports are access ports. To workaround this issue with the VNXe we changed the switches trunk ports to the VNXe to use VLAN 2 as the native VLAN and set the VNXe to not tag the traffic for that VLAN. It has been working well for the last week or so with heavy traffic the last 2 days.
jjgrinwis
48 Posts
0
December 2nd, 2011 04:00
There is an official bug-id for this issue: AR 456370
Should be solved in a next release of the VNXe code.
Richard_Dagenai
3 Posts
0
December 6th, 2011 17:00
Yes John, tcpdump is available on the VNXe. However this tool is only available to root so you will need to call support to get access to it. They will set up a webex and inject a service tool that allows them to interact with the system as root. You will find them cooperative in getting a tcpdump going.