This post is more than 5 years old
86 Posts
2
6856
VNXe NFS & iSCSI Interface Configuration
I'm working with a VNXe 3100 that is utilizing both block iSCSI and NFS for attached VMware ESXi hosts. Up until now, they have only used NFS, but we need to use iSCSI additionally. I have read all kinds of EMC documentation & internet blogs on the subject of multipathing & best practices, but none of them seem to address a configuration where BOTH NFS & iSCSI are in use simultaneously.
For example, in the EMC VNXe High Availability white paper, it talks about configuring iSCSI & then configuring NFS. In both scenarios, they seem to be using the same physical SP ports for both block & file traffic. In my opinion, this wouldn't be best practice since its usually best to isolate iSCSI traffic.In our environment, we have redundant switches between the SP's & ESXi hosts. My thought is that there should be a seperate VLAN & physical interface for iSCSI and seperate VLAN and interfaces for NFS. We are only using 1Gb ports. Thoughts?
I'd love to either see documentation or someone's firsthand experience on how this is accomplished according to best practices. Let me know if you need more details. Thanks in advance.
chrislogo
98 Posts
0
April 5th, 2014 02:00
Hello Clayton,
Not sure if you have gone through the attached document it could be helpful.
Best practice would be to use different set of eth ports from different SPs
Assuming that you have extra IO Module the configuration would be to use ex - eth2 and eth3 for NFS on SPA and eth 10 and eth11 for iSCI on SPB , or any combination of eth ports and SPs as per your configuration.
Chris
1 Attachment
h10599-multipathing-path-redundancy-vnxe-abpg.pdf
clayman88
86 Posts
0
April 15th, 2014 12:00
Thanks, chrislogo. I believe that answered the question. I didn't have that document. In this scenario, we've got no additional ports available, so I'll plan on purchasing an additional 4-port 1Gb ethernet module for the VNXe.
In our scenario we would do the following I believe:
SPA:
eth2 & eth3 - LACP to switch for NFS
eth10 & eth11 - LACP to switch for iSCSI
SPB:
eth2 & eth3 - LACP to switch for NFS
eth10 & eth11 - LACP to switch for iSCSI
Sound right?
chrislogo
98 Posts
0
April 16th, 2014 02:00
NFS looks good with LACP , use eth2 on switch A and eth3 on switch B , using stacked switches , more on the HA document. This will give you switch redundancy.
For iSCSI the general practice is not to use LACP but connect eth10 to switch A on subnet A and eth11 to switch B on subnet B
This will give you switch redundancy
Chris
clayman88
86 Posts
0
April 16th, 2014 08:00
Based on that document, it sounds like they are recommending a seperate iSCSI VLAN/Subnet for each SP, not each port. In other words, 10 & 11 on SPA would be the same subnet. Am I missing something?
chrislogo
98 Posts
1
April 16th, 2014 08:00
The document provides both the scenarios , ISCSI Config with and without LACP and the respective advantages and disadvantages.
The HA document on page 14 talks about iSCSI Config with separate subnets for eth2 and eth3
You can look into both the scenarios and configure your environment accordingly..
Chris
1 Attachment
docu35554_White-Paper_-VNXe-High-Availability-VNXe3100,-VNXe3150,-and-VNXe3300-Overview.pdf
clayman88
86 Posts
0
April 16th, 2014 09:00
Based on my testing with our VNXe, it doesn't sound like its possible to create more than one LAG on the same module. Ideally, I want two LAGs per SP. I've attached a diagram of what I'm thinking. Thoughts?
chrislogo
98 Posts
1
April 16th, 2014 23:00
Yes will not be able to create two LACP on a single Module
You will have to combine eth ports from different Modules
Example - eth2 and eth3 can be one LACP and eth10 and eth11 from the second module can be in an LACP config
You cannot LACP between eth4 and eth5 on the same module . All LACP config should be with the base port eth2 or eth10(incase of second module)
So as examples eth2 and eth3 can be LACP, eth,2 eth3 and eth4 can be an LACP ,, but eth3 and eth4 cannot be an LACP
Chris
clayman88
86 Posts
0
April 17th, 2014 06:00
I see! Well that explains it then. So, here is hte deal. There are only two modules, a 2-port 1Gb Ethernet (which is totally in use for NFS) and a 4-port 1Gb Ethernet which will be dedicated to iSCSI. Having said that, I won't be able to use multiple modules. How does this sound? The corresponding ports on teh switch stack will be LAG'd for each VLAN/Subnet. No routing.
clayman88
86 Posts
0
May 20th, 2014 07:00
Does that make sense? Any suggestions?
Shardul1
33 Posts
0
May 20th, 2014 08:00
Hello Clayton,
Like Chris, I would also discourage setting up LACP (aggregation) for iSCSI ports. I have seen instances where failover happens extremely slow when iSCSI sits on an aggregation.
On the last figure, same port number for SPA and SPB should not be on different subnets/vlan. The thumb rule is if ethX of SPA goes to vlanA/subnet A, then ethX of SPB should also be connected to vlanA/Subnet A.
I would connect NFS over the 4 port module using LACP, and have iSCSI multipathing (which is similar to port binding on ESX) on the other 2 port module.
Shardul
clayman88
86 Posts
0
May 20th, 2014 09:00
Thanks, Shardul. The 2-port module is already in use for NFS and therefore our only option is to use the 4-port 1Gb modules.
I think I should probably start over and explain this again because I'm having a hard time wrapping my head around this & everywhere I look, I'm getting conflicting information.
Let's completely forget about the NFS configuration for now & only focus on iSCSI. NFS is alrady in place on the integrated 2-port module & is not changing. We have a 4-port 1Gb module that will be used solely for iSCSI connectivity. Upstream is a pair of Brocade Stacked switches.
What I need to find out is what is best practice for connecting iSCSI from the VNXe through a pair of stacked switches to some ESXi hosts. Should I use a LAG or just create 4 separate networks, one for each port?
chrislogo
98 Posts
0
May 20th, 2014 10:00
Ideal configuration would be
eth10 from both SPA and SPB go to Switch 1 Subnet 1 and
eth11 from both SPA and SPB go to Switch 2 Subnet 2
Reference page 14 from HA doc , please note the switch used in this are 'not' Stacked Switches.
Chris
clayman88
86 Posts
0
May 20th, 2014 10:00
Assuming I only use 2 ports on either SP and I don't do any link aggregation, here is what I've come up with. Thoughts?
clayman88
86 Posts
0
May 20th, 2014 12:00
Chris, I get that & I appreciate it, but that is not taking into account that this is a stacked switch. Does that change anything?
Shardul1
33 Posts
0
May 21st, 2014 00:00
The major advantage of stacked switches is the ability to do cross stack aggregation.
As we wont be doing LACP for iSCSI, I concur with Chris' suggestion. Having that configuration, you will always have an available path to each SP even if one of the switches go down.