Are you sure you are using the correct interfaces and SFP’s on the Cisco ports connecting to the VNX 10Gbit ports ?

We only support 10GBASE-SR – 850nm short-range with multi-mode fibers like used with Cisco XENPAK-10GB-SR

Rainer

I would suggest to try generating a procedure with the VNX prodecure generator on Powerlink and follow that.

I cant attach a doc here - I'll put it into documents temporarily

https://community.emc.com/docs/DOC-13150

Rainer

Hi Rainer, thanks for replying. The SFP in the switch is SFP-10G-SR - Cisco 10GBase-SR SFP Module. We did have someone from EMC verify the compatibility of it. They said it was supported.

Thank you. I'll give that a try.

Were you ever able to get this resolved?

By chance are the Management Ports and the iSCSI ports on the array using the same subnet?

Are you using Jumbo frames? 10Gb on VNS only supports 4000 MTU.

Are you using flow control on the swtiches?

glen

"Are you using Jumbo frames? 10Gb on VNS only supports 4000 MTU."

wait a minute... do you mean only when Jumbo frames are enabled? or in general?

Without Jumbo frames Ethernet max MTU is around 1500

I assume you know that within a VLAN all nodes have to have the same Jumbo setting

Ok , yes that's clear. I thought something funny was going on with the default max MTU size of the VNX.

But still, 4000 is the max? i thought 9000 was the common configuration for jumbo frames.

The default is 1500, the max is 4000 - that's what engineering set it to.

Also, I've seen some discussions on the ESX blogs where someone tested the advantage of 9000 MTU and found that it really did not provide that much of a benefit:

http://www.boche.net/blog/index.php/2011/01/24/jumbo-frames-comparison-testing-with-ip-storage-and-vmotion/

glen

According to the "EMC Unified Storage Best Practices for Performance and Availability -- Common Platforum and Block Storage 31.0 -- Applied Best Practices" from 6/23/2011 on page 28:

"The VNX series supports 4000, 4080, or 4470 MTUs for its front-end iSCSI ports.  It is not recommended to set your storage network for Jumbo frames to be any larger than these."

In version 31.5 released 10/17/2011:

"The VNX support front end iSCSI port MTU sizes of: 1260, 1448, 1500, 1548, 2000, 2450, 3000, 4000, 4080, 4470, 5000, 6000, 7000, 8000, 9000.  it is not recommended to set your storage network for Jumbo frames to be any other than these." 

With that said, I went with 4470 when we deployed our two VNX5300's a few months ago when the 31.0 document was released.  So, what's the official word?

Thanks for the clarification.  Surely you don't expect me to read the first page of the document instead of just Ctrl-F and search for MTU.    *cough*

I missed it also, until I searched for "jumbo"

glen

from the "EMC Unified Storage Best Practices for Performance and Availability - Common Platform and Block Storage 31.5 — Applied Best Practices.pdf" on page 5 - this was changed in this version of the document. So the settings in the 31.5 are the ones to follow - it goes along with the changes in flare 05.31.000.5.5xx. So if you have an older flare version, you need to follow the original recommendations. With the new flare, follow the recommendations in the 31.5 document:

glen

>So, what's the official word?

What is the question?

Between Block O.E.31.0 and Block O.E. 31.5 releases of VNX BP, it was pointed out that CX-series Jumbo Frame values were being used for VNX.  That section was corrected to be VNX. 

The numbers are so specific, because the size increments for Jumbo Frames varies between vendors.  The listed values are the values the VNX's front-end iSCSI ports support.  Correlate your iSCSI HBA Jumbo Frame available settings, with your Switches Jumbo Frame settings, and the VNX's front-end iSCSI port Jumbo Frame settings.

Aw Glen, you just about re-wrote that section yourself.