Unsolved
This post is more than 5 years old
4 Posts
0
120953
iscsi vlan setup using 5424
Greetings,
I apologize in advance if this question has been asked a thousand times before but I'm just starting to get my feet wet with vlans and everything is still pretty abstract to me and I'm not sure if the answers from the other thread is applicable to what I am trying to do. I'm trying to create an iscsi vlan that spans two switches, I'm thinking about getting a couple of 5424s for this. This is how I am envisioning it:
A couple of questions first, in the setup above, if I connect both port 1 to my main switch, ports 2-12 will become part of the local main network right? Ports 13-20 will be it's own lan and will not receive broadcasts from the main network? Are four ports enough / too-few / too-much for interconnect between the two switches?
I have read this helpful page, but I am not sure how to implement the vlan with the instructions provided, I think the vlan setup is just another procedure on top of of the setup shown in that page.
Although I still don't understand most of how vlans work, I feel I'm getting close to what I am trying to do, could someone help me understand this a bit more?
Thanks in advance.
C
call-in
4 Posts
0
December 16th, 2010 11:00
So after a couple of days without a reply I'm starting to think that what I'm trying to do is not possible, am I going about this the wrong way? If so, then what would be the best way to use a couple of 5424s to switch iSCSI traffic on some ports and make the rest part of our main LAN?
Anybody?
call-in
4 Posts
0
December 16th, 2010 18:00
Okay, so here is my stab at this, will this config set on both switches do what I want it to?
console# sh ru
spanning-tree mode rstp
interface range ethernet all
spanning-tree portfast
exit
interface range ethernet all
flowcontrol on
exit
port jumbo-frame
vlan database
vlan 2
exit
interface range ethernet g(13-20)
switchport access vlan 2
exit
interface port-channel 1
switchport access vlan 2
exit
interface vlan 2
name "iSCSI traffic"
exit
interface range ethernet g(21-24)
channel-group 2 mode on
exit
iscsi target port 860 address 0.0.0.0
iscsi target port 3260 address 0.0.0.0
iscsi target port 9876 address 0.0.0.0
iscsi target port 20002 address 0.0.0.0
iscsi target port 20003 address 0.0.0.0
iscsi target port 25555 address 0.0.0.0
no iscsi enable
interface vlan 1
ip address 192.168.0.xxx 255.255.255.0
exit
ip default-gateway 192.168.0.1
username xxxxxxxxxxxxx password ffffffffffffffffffffffffffffffff level 15 encrypted
snmp-server community Dell_Network_Manager rw view DefaultSuper
sntp client enable vlan 1
Default settings:
Service tag: XXXXXXX
SW version 2.0.0.43 (date 02-Sep-2010 time 09:01:52)
Gigabit Ethernet Ports
=============================
no shutdown
speed 1000
duplex full
negotiation
flow-control off
mdix auto
no back-pressure
interface vlan 1
interface port-channel 1 - 8
spanning-tree
spanning-tree mode STP
qos basic
qos trust cos
console#
I hope someone can help me.
Joespower
11 Posts
0
July 23rd, 2012 10:00
Call-in, did you ever get this setup to work? I'm tasked with setting up something similar in my environment for use with a NetApp filer, but I'm having trouble finding guidance on the configuration.
DELL-Willy M
802 Posts
1
July 23rd, 2012 11:00
Joespower,
We can help with any questions you might have. Can you provide more specific information on what you are configuring and what hardware is involved?
Joespower
11 Posts
0
July 23rd, 2012 12:00
Thanks Willy,
I have two 5424 switches that I'm going to use as my SAN. Originally, we thought we were going to be purchasing an Equillogic PS6100, so we set these up as pure iSCSI switches with a LAG and didn't even bother with a port for web-based config. In the end, our applications dictated that we needed a filer that supported multiple protocols, so we went with the NetApp 2240-2. Now, I have to reconfigure these switches to support both iSCSI and NFS traffic, I'm assuming on seperate VLANs with both VLANs utilizing the LAG. I initially thought about using all ports to do either iSCSI or NFS depending on the VMs on a given host, but I can't find any config examples like that. I have networking experience, but I definately feel like I'm in over my head on this one, so any advice is welcome. Here is a picture of what I assume the end result should be:
And here is the current config from back when we were looking to do pure iSCSI:
Joespower
11 Posts
0
July 23rd, 2012 13:00
Thanks for the quick response Willy...
I have seen the article that you refer to linked many times in this forum, and its quite helpful. Also, your post does a good job of describing how to configure the picture I sent. There is another alternative though that might be more appealing to us, and I can't find any instructions on how to achieve it here. See, the inherent problem with the picture I sent is that your VM hosts would each have to have 2 ports for NFS traffic and 2 ports for iSCSI traffic (2 protocols across 2 switches), but what if I only have 2 ports available per host for the SAN? Is it possible to to group up the first 18 ports into 2 VLANs such that you have a VLAN for NFS **AND** a VLAN for iSCSI on a given port? The traffic would still be segregated and could have different configuration options, and I would need only 2 physical NICs for SAN per host. If this is possible, how would I accomplish it, and wouldn't I have to optimize iSCSI in this scenario? See the picture...
Also, feel free to talk me out of doing it this way if you foresee me having problems...
DELL-Willy M
802 Posts
1
July 23rd, 2012 13:00
Here is a good document for Configuring a PowerConnect 5424 or 5448 Switch for use with an iSCSI storage system.
en.community.dell.com/.../configuring-a-powerconnect-5424-or-5448-switch-for-use-with-an-iscsi-storage-system.aspx
I would look at the config on the document above and mold that to the ISCI ports. You would need port jumbo frame, flowcontrol on spanning tree portfast on the ISCI ports. Then configure the LAG choose Static of Dynamic (LACP).
Creating LAGs – Static Two-port LAG
console> enable
console# configure
console(config)# interface range ethernet g1-2 (adjust for the 4 ports bonding the SAN switches together)
console(config-if)# channel-group 1 mode on
console(config-if)# exit
console(config)#
Creating LAGs – Dynamic LACP LAG
console> enable
console# configure
console(config)# interface range ethernet g1-2
console(config-if)# channel-group 1 mode auto
console(config-if)# exit
console(config)#
On the LAG port-channel or uplinks you will need to create a trunk/general switchport. This will allow multiple VLANs to traverse back and forth between the switches. Depending on the compatibility of your devices you will use trunk or general.
Configuring Trunk Ports
console> enable
console# config
console(config)# interface ethernet 1/e5
console(config-if)# switchport mode trunk
console(config-if)# switchport trunk allowed vlan add 2
console(config-if)# end
You can set up the VLAN like this
console> enable
console# config
console(config)# vlan database
console(config-vlan)# vlan 2 (Optional VLAN numbers)
console(config-vlan)# vlan 3
console(config-vlan)# exit
console(config)# interface vlan 2
console(config-if)# name ISCI Traffic (Optional Naming)
console(config)# interface vlan 3
console(config-if)# name NFS Traffic
Then any ports that you want to connect to VLAN 2 would need this configuration. You can use the range command for multiple ports. Then a similar set up for the ports in VLAN 3.
Configuring Access Ports
console> enable
console# config
console(config)# interface ethernet 1/e1
console(config-if)# switchport mode access
console(config-if)# switchport access vlan 2
console(config-if)# end
Hope this helps,
Keep us updated if you can.
Joespower
11 Posts
0
July 23rd, 2012 15:00
Here is the config that I've come up with for the first option, with NFS and iSCSI relegated to separate physical ports on the switches. Remember that in this config, port 19 will remain on VLAN 1 for management. I'm 99% sure of all of this setup with the exception of creating the trunking LAG and enabling portfast/rstp on it. Please advise...
DELL-Willy M
802 Posts
1
July 23rd, 2012 16:00
You would not enable portfast on the LAG Trunk, that should be with the access mode commands along with flowcontrol.
console(config)# interface range ethernet g(11-18)
console(config-if)# switchport mode access
console(config-if)# switchport access vlan 200
console(config-if)# spanning-tree portfast
console(config-if)# flowcontrol on
On the 4 port LAG do not use any spanning tree commands.
If your vlan traffic is needing to reach to the core you will need trunk/general settings on that port also where VLAN 2 is.
Additional info:
8 Unicast Storm Control
A traffic “storm” occurs when a large outpouring of packets creates excessive network traffic that degrades network performance. Many switches have traffic storm control features that prevent ports from being disrupted by broadcast, multicast, or unicast traffic storms on physical interfaces. These features typically work by discarding network packets when the traffic on an interface reaches a percentage of the overall load (usually 80 percent, by default).
Because iSCSI traffic is unicast traffic and can typically utilize the entire link, it is recommended that you disable unicast storm control on switches that handle iSCSI traffic. However, the use of broadcast and multicast storm control is encouraged.
The following example for Dell PowerConnect 5400 series switch cover disabling Unicast Storm Control on the user specified interfaces responsible for passing iSCSI traffic. They begin assuming the user is in privileged EXEC mode, and disable unicast storm control on local switch port 1.
Dell PowerConnect 5400 Series CLI Example:
# configure
# interface ethernet g1
# no port storm-control include-multicast
# end
Answering you 2nd scenario:
If you have a virtual switch on your VM host you may be able to set up trunk/general mode on both sides where multiple vlans can traverse. The packets need to be tagged at some point and that is where it gets tricky. The switchport access mode is what tags traffic for a certain VLAN. There is not much documentation on this. That said the setup we have laid out above is the more utilized and recommend option for isolating traffic into different VLANs.
Joespower
11 Posts
0
July 23rd, 2012 16:00
Willy, you are the best! It would have taken me MUCH longer to get responses like this from Cisco (I know from experience). I am kind of bummed that I need to buy more NICs, but I can live with it if it means stuff will just work.
Concerning VLAN 2, I see your point. The purpose of that port will be to replicate data between remote SANs over a 10Mb connection linked to our core switches. I don't yet know what protocol that replication traffic will require, but lets assume that its just accessing the volumes as NFS shares. In that case, are you saying I would have to create a trunk between the core switches and the SAN switches and allow only VLAN 100 over that trunk?
I will try out these suggestions and reply here with any questions/problems...
Again, thanks!!
DELL-Willy M
802 Posts
1
July 24th, 2012 10:00
Yes, Any time you are connecting 2 switches/routers you are going to need a trunk to allow the VLAN (Layer 2) traffic to move back and forth.
It was a pleasure working with you.
Joespower
11 Posts
0
July 24th, 2012 11:00
OK, this is my new config mock-up based on our discussion: (see my questions after)
RESET SWITCH TO FACTORY
enable
delete startup-config
y
reload
y
y
DO NOT ENTER THE SETUP WIZARD
n
BEGIN PROGRAMMING
enable
configure
hostname SANx
REMOVE ALL VOIP JUNK
voice vlan oui-table remove 00036b
voice vlan oui-table remove 0001e3
voice vlan oui-table remove 000fe2
voice vlan oui-table remove 0060b9
voice vlan oui-table remove 00d01e
voice vlan oui-table remove 00e075
voice vlan oui-table remove 00e0bb
voice vlan oui-table remove 00096e
DISABLE ISCSI OPT
no iscsi enable
no iscsi target port 860
no iscsi target port 3260
no iscsi target port 9876
no iscsi target port 20002
no iscsi target port 20003
no iscsi target port 25555
DO SOME GLOBAL LEVEL CONFIGURATION
spanning-tree mode rstp
port jumbo-frame
CONFIGURE ETH PORTS
interface range ethernet all
speed 1000
duplex full
exit
CREATE VLAN 2 (UPLINK TO CORE)
vlan database
vlan 2
exit
interface vlan 2
name "CORE UPLINK"
exit
CREATE VLAN 100 (NFS TRAFFIC)
vlan database
vlan 100
exit
interface vlan 100
name "NFS TRAFFIC"
exit
CREATE VLAN 200 (ISCSI TRAFFIC)
vlan database
vlan 200
exit
interface vlan 200
name "ISCSI TRAFFIC"
exit
ASSIGN PORTS TO VLAN 2
interface range ethernet g20
switchport mode trunk
switchport trunk allow vlan add 100
exit
ASSIGN PORTS TO VLAN 100
interface range ethernet g(1-8)
switchport mode access
switchport access vlan 100
exit
ASSIGN PORTS TO VLAN 200
interface range ethernet g(11-18)
switchport mode access
switchport access vlan 200
spanning-tree portfast
flowcontrol on
no port storm-control include-multicast
exit
CREATE 4-PORT TRUNKING LAG AND ADD VLAN 100 AND 200
interface range ethernet g(21-24)
channel-group 1 mode on
exit
interface port-channel 1
switchport mode trunk
switchport trunk allow vlan add 100 200
flowcontrol on
exit
SAVE CONFIG AND REBOOT
exit
copy running-config startup-config
reload
Questions:
1) Is it OK to set RSTP globally on all ports if we're going to later use portfast for the iSCSI ports?
2) Portfast, flowcontrol, and storm-control should not be necessary on any VLAN other than 200, correct? In other words, these are best practices for iSCSI when NOT optimized and therefore only necessary for the ISCSI VLAN...
3) Do I still need to enable flowcontrol on the LAG as I have?
4) Any other issues you can see?
DELL-Willy M
802 Posts
1
July 24th, 2012 12:00
Here is some additional best practice information for ISCI
www.dell.com/.../ip-san-best-practices-en.pdf
Ether Flow Control: Dell recommends that you enable Flow Control on the switch ports that handle iSCSI traffic. In addition, if a server is using a software iSCSI initiator and NIC combination to handle iSCSI traffic, you must also enable Flow Control on the NICs to obtain the performance benefit. On many networks, there can be an imbalance in the network traffic between the devices that send network traffic and the devices that receive the traffic. This is often the case in SAN configurations in which many hosts (initiators) are communicating with storage devices. If senders transmit data simultaneously, they may exceed the throughput capacity of the receiver. When this occurs, the receiver may drop packets, forcing senders to retransmit the data after a delay. Although this will not result in any loss of data, latency will increase because of the retransmissions, and I/O performance will degrade.
Spanning-Tree Protocol: It is recommended that you disable spanning-tree protocol (STP) on the switch ports that connect end nodes (iSCSI initiators and storage array network interfaces). If you still decide to enable STP on those switch ports, then you should turn on the STP FastPort feature on the ports in order to allow immediate transition of the ports into forwarding state. (Note: FastPort immediately transitions the port into STP forwarding mode upon linkup. The port still participates in STP. So if the port is to be a part of the loop, the port eventually transitions into STP blocking mode.)
Note: PowerConnect Switches default to RSTP (Rapid Spanning Tree Protocol) an evolution in STP that provides for faster Spanning tree convergence and is preferable to STP
Note: The use of Spanning-Tree for a single-cable connection between switches or the use of trunking for multiple-cable connections between switches is encouraged.
1) Your preference on what is best for your environment
2) Portfast, flowcontrol, and storm-control in this discussion is only needed for the ISCSI vlan 200
3) Flow control would need to be enabled on any port/lag that has ISCSI traffic