SAN using 2 x 5424s to a NetApp FAS2240

Question

Hi experts! I am having a difficult time figuring out how to configure my 2 PC 5424s for use in a SAN.  Connected to these switches is a NetApp FAS 2240 with redundant controllers, 4 NIC ports each.  On the other end I have 6 VM hosts (various Dell servers), also with 4 NICs each.  As a guide, I'm using the NetApp Storage Best Practices for VMware vSphere, and it states pretty clearly that 'switching capabilities determine the storage network architecture' (section 3.5).  Basically, I can do it one way if my physical switches will do MSLA, or I can do it another way for traditional switches.  I've been able to determine that these switches can't do MSLA, so I've tried to follow the instructions for the traditional switches (section 3.7), but its a little vague.  It basically tells me that, if my virtual network is built correctly, 'switch-side configuration is not required.'  But, when I've posted my virtual network up on the Vmware forums, they still give me extra configuration steps, like segregating traffic using VLANs, etc.  I'm going to post a drawing of my virtual network here, then I'll have some additional questions... So, as you can see, the NetApp plan has me segregating traffic essentially using different subnets only.  They mention VLANs in the document, but its more about segmenting storage traffic from LAN traffic on the same switches.  I have them physically separated.  But, my 2 5424s are the 2 SAN switches pictured, and they will have NFS, iSCSI, and Vmotion traffic flowing through them.  All of these are on separate subnets, but I've had others suggest to use VLANs here even though NetApp says I shouldn't have to configure the physical switches.  So, here are my questions to you guys, the experts: 1) Do I need to have tagged VLANs in this scenario, and why?2) Since this is a 'non-Etherchannel' solution and my multipathing is being handled by vSphere, do I really need any sort of trunk/LAG between switches?  I'm confused because the wording of the NetApp document (section 3.7) seems to indicate that I don't need it, but they have the 2 switches LAGged in every example picture in the entire chapter.  What would LAGging accomplish in this scenario?3) Would it be beneficial to LAG the ports going to the NetApp controllers, or would that too be handled by IP?  Again, the document seems to suggest that I should LAG the controllers, but it also says I shouldn't have to configure the switches...4) How does iSCSI optimization come in to play here?  What about RSTP?  What type of configuration (besides jumbo frames) do the SAN switches need? I've been working on understanding this for the better part of a month now, and I still haven't done any actual configuration.  If I could afford Cisco switches, this would have taken 48 hours tops!  I appreciate any clarification that anyone can offer.  In fact, I would be willing to buy someone a beer (or soda on the off chance that my savior is under aged)... Thanks!

Joespower · Answer

Wow, I really expected I would get a response on this. Is there not anyone using these or similar switches with NetApp equipment?

Here's a pic of what I think the port breakdown should be:

Any takers?

DELL-Willy M · Answer

1) If you are needing to keep the traffic segregated on the switch then VLANs would be needed. You can set up trunk/general switchport mode on the connection to allow multiple VLANs in and out of the 5424 switches. Page 17 of the User Guide discusses VLAN support features.

support.dell.com/.../en_ug.pdf

2,3) LAG is Link Aggregation. Where you have multiple physical connections grouped together to form a single link. You can choose to trunk the connection or not . Most do trunk because they group multiple VLANs together to reach another VLAN aware device and need greater throughput and redundancy on the connection. When choosing whether to LAG or not especially in a storage environment you need to think about the speed and throughput you have on every connection along the path from the server to the storage device and provide the same speed and throughput in order to prevent and bottlenecks or possible latency on the network.

4) Here are a couple documents discussing iSCSI optimization:

www.dell.com/.../ip-san-best-practices-en.pdf

en.community.dell.com/.../configuring-a-powerconnect-5424-or-5448-switch-for-use-with-an-iscsi-storage-system.aspx

Hope this helps,

Keep us updated if you can.

Joespower · Answer

Thanks for the reply Willy!

I've read through the docs you referenced. The difficult thing when dealing with these switches is that, since all of Dell's SMB SAN offerings are iSCSI only, almost everything I read on the Dell forums tells me to physically isolate iSCSI switches or switch ports. Thats makes things easier, but in my case, I can't really afford to do that. So, based on what you've given me to read, let me touch on these questions again...

1) I accept that I need to use tagged VLANs even though I'm using separate subnets. If you tell me and the Vmware guys tell the same thing, I can be pretty sure its true. So, does that mean that, based on the drawings above, I need to make the first 14 ports above ONLY accept tagged VLAN traffic and tag all traffic coming in from the virtual network?

2) Assuming the first question's answer is "Yes," I would put the first 14 ports into general mode, and the last 9 ports would be in trunking mode (3 sperate trunks of course)?

3) Since I'm going to be segregating the traffic with VLANs, does iSCSI optimization come into play? I realize I will have both iSCSI and non-iSCSI frames going over the same switchports, but if they're logically separate, what does optimization accomplish? Its difficult to discern this from the documents you referenced...

I will post a sample config in a little while...

Joespower · Answer

Below is the CLI config based on everything I understand at this point. A few questions and things to keep in mind:

1) port 15 will be on the default VLAN for MGMT and doesn't get any configuration.
2) I have enabled iSCSI optimization on port 3260, but I haven't set any parameters since I don't yet understand them. Am I still getting optimized witht the default settings?
3) In the section labeled "ASSIGN PORTS TO VLANs 50, 100, & 200" I'm essentially adding optimized setting meant for iSCSI to all ports (eg portfast, flowcontrol, storm-control). Will any of that adversely effect the other traffic on these ports? Is there are way to isolate these settings to VLAN 200 only?
4) In the "LAG TO STORAGE DEVICE" section, should that be a trunk like the link between switches, or would that be a general mode port-channel?

Here is the config:

RESET SWITCH TO FACTORY

enable
delete startup-config
y
reload
y
y

DO NOT ENTER THE SETUP WIZARD

n

BEGIN PROGRAMMING

enable
configure
hostname SANx

REMOVE ALL VOIP JUNK

voice vlan oui-table remove 00036b
voice vlan oui-table remove 0001e3
voice vlan oui-table remove 000fe2
voice vlan oui-table remove 0060b9
voice vlan oui-table remove 00d01e
voice vlan oui-table remove 00e075
voice vlan oui-table remove 00e0bb
voice vlan oui-table remove 00096e

DO SOME GLOBAL LEVEL CONFIGURATION

spanning-tree mode rstp
port jumbo-frame

ENABLE ISCSI OPT ON PORT 3260

iscsi enable
iscsi target port 3260

no iscsi target port 860
no iscsi target port 9876
no iscsi target port 20002
no iscsi target port 20003
no iscsi target port 25555

CREATE VLAN 20 (UPLINK TO CORE)

vlan database
vlan 20
exit
interface vlan 20
name "CORE UPLINK"
exit

CREATE VLAN 50 (VMOTION TRAFFIC)

vlan database
vlan 50
exit
interface vlan 50
name "VMOTION TRAFFIC"
exit

CREATE VLAN 100 (NFS TRAFFIC)

vlan database
vlan 100
exit
interface vlan 100
name "NFS TRAFFIC"
exit

CREATE VLAN 200 (ISCSI TRAFFIC)

vlan database
vlan 200
exit
interface vlan 200
name "ISCSI TRAFFIC"
exit

ASSIGN PORTS TO VLAN 20

interface range ethernet g16
switchport mode trunk
switchport trunk allow vlan add 100,200
exit

ASSIGN PORTS TO VLANs 50, 100, & 200

interface range ethernet g(1-14)
switchport mode general
switchport general acceptable-frame-types tagged-only
switchport general allowed vlan add 50,100,200 tagged
spanning-tree portfast
flowcontrol on
no port storm-control include-multicast
exit

CREATE 4-PORT TRUNKING LAG TO STORAGE DEVICE
AND ADD VLAN 50, 100 AND 200

interface range ethernet g(17-20)
channel-group 1 mode on
exit
interface port-channel 1
switchport mode trunk
switchport trunk allow vlan add 50 100 200
flowcontrol on
exit

CREATE 4-PORT TRUNKING LAG BETWEEN SWITCHES
AND ADD VLAN 50, 100 AND 200

interface range ethernet g(21-24)
channel-group 1 mode on
exit
interface port-channel 1
switchport mode trunk
switchport trunk allow vlan add 50 100 200
flowcontrol on
exit

SAVE CONFIG AND REBOOT

exit
copy running-config startup-config
reload

DELL-Willy M · Answer

2) Page 232 starts the discussion about iSCSI Optimiaztion.

support.dell.com/.../en_ug.pdf

3) I personally do not know of any adverse affects for the other traffic coming into the port. I do not see a way to isolate the settings to a VLAN it is a per port basis.

4) It really depends on how your storage device handles the compatible connection type.

Also you are probably going to want portfast set up on all the ports or LAGs that are handling storage or iSCSI traffic.

console(config-if)# spanning-tree portfast

Networking General

Was this post helpful?