This post is more than 5 years old
5 Posts
0
319163
Dell MD3200i correct setup with IP Scheme and network map for faster Network speeds
Hello Everyone,
I have been working on this project for a while now. I have done lots of research and have contacted Dell multiple times. I need some help.
I have 2 sets of identical servers to be used for Hyper-V clusters. We purchased a Dell MD3200i in November, we also purchased a Dell 5424 switch and a Cisco SLM2048 switch. These servers have Real Tek network cards 2 each for iSCSI, Jumbo frames is on to 9K
The Dell MD3200i is dual controller SAN. The Dell 5424 is the iSCSI optimized switch and the Cisco is a 48 port switch. iSCSI is on its own network nothing else is connected to it. (There is a Iomega NAS and a Promise Tech Vtrak SAN with the domain controller all running with iSCSI)
Dell MD3200i Controller 0 Port 0 192.168.130.101 Controller 1 Port 0 192.168.131.101
Controller 0 Port 1 192.168.130.102 Controller 1 Port 1 192.168.131.102
Controller 0 Port 2 192.168.130.103 Controller 1 Port 2 192.168.131.103
Controller 0 Port 3 192.168.130.103 Controller 1 Port 3 192.168.131.104
Cisco Switch has 1 VLAN #1 on it and only the 130 subnet items in it. There are 8 connections to the switch. 4 from Controller 0 and 1 from each of the servers. This has an IP address of 192.168.132.4 but that switch IP should not make any difference. Jumbo frames turned on
Dell Switch has 2 VLAN's #1 for the misc stuff like the DC, Vtrack and the NAS. Subnet 2 is 131 and has a total of 8 connectors in it. 4 from Controller 1, and 1 each from the servers. Jumbo frames turned on. IP Address of the switch is 192.168.132.2
Server Groups. Each Server node is running Windows 2008 R2 SP1 and running hyper-v clusters with Virtual Servers / file storage on them. 2 Realtek NIC's with Jumbo frames turned on to 9K ProdNode1 and ProdNode2 have 12 Core i7 CPU with 12 GB Memory, EXNode1 and ExNode2 have 12 Core i7 CPU with 24 GB memory.
ProdNode1 iSCSI-1 192.168.130.110 ProdNode1 iSCSI-2 192.168.131.110
ProdNode2 iSCSI-1 192.168.130.120 ProdNode2 iSCSI-2 192.168.131.120
ExNode1 iSCSI-1 192.168.130.130 ExNode1 iSCSI-2 192.168.131.130
ExNode2 iSCSI-1 192.168.130.140 ExNode2 iSCSI-2 192.168.131.140
iSCSI Discovery portal on ProdNode1 went to 192.168.130.101 from 192.168.130.110 and 192.168.131.101 from 192.168.131.110 Each of the other servers discovery portal were pairs like that.
iSCSI Targets Tab click that, add 8 sessions MPIO to each of the ports on the Dell MD3200i. For example I added 1 session from 192.168.130.101 using this. Add Session, --> Enable Mutlipath --> Advanced --> Local Adapter iSCSI --> Initiator IP 192.168.130.110 and Target IP 192.168.131.101. Repeated this on all servers. 4 Connections for each NIC.
The network is very slow. It was running around 2 to 4 MB / MIN and it should be much faster. At first we only had the Cisco Switch with only 1 VLAN and everything connected directly to it. I read somewhere that MPIO was the problem so I went through the same setup but did not check MPIO checkbox above. Once it was done, the network speed increased to 1.5 GB / Min but the Dell Modular storage wasn't happy and the Virtual servers would fail and not very stable connection.
What is wrong with my setup? Can I do direct attached and attach the Servers directly to the PowerVault and will that make it stable and allow for failover and NIC redundancy? I need to get this stable so I can move on with something else.
Thanks Ron
Dev Mgr
9.3K Posts
1
April 14th, 2011 07:00
You will want to change your IP scheme to this:
Your iSCSI sessions will then be:
For EX-Node1:
- 192.168.130.130 to 192.168.130.101
- 192.168.130.130 to 192.168.130.102
- 192.168.131.130 to 192.168.131.101
- 192.168.131.130 to 192.168.131.102
For EX-Node2:
- 192.168.132.140 to 192.168.132.101
- 192.168.132.140 to 192.168.132.102
- 192.168.133.140 to 192.168.133.101
- 192.168.133.140 to 192.168.133.102
For Prod-Node1:
- 192.168.130.10 to 192.168.130.101
- 192.168.130.10 to 192.168.130.102
- 192.168.131.10 to 192.168.131.101
- 192.168.131.10 to 192.168.131.102
For Prod-Node2:
- 192.168.132.20 to 192.168.132.101
- 192.168.132.20 to 192.168.132.102
- 192.168.133.20 to 192.168.133.101
- 192.168.133.20 to 192.168.133.102
Dev Mgr
9.3K Posts
0
April 9th, 2011 20:00
First, change your IP scheme back to the factory default; it was set like that for a good reason; it's the best practice:
controller 0 port 0: 192.168.130.101
controller 0 port 1: 192.168.131.101
controller 0 port 2: 192.168.132.101
controller 0 port 3: 192.168.133.101
controller 1 port 0: 192.168.130.102
controller 1 port 1: 192.168.131.102
controller 1 port 2: 192.168.132.102
controller 1 port 3: 192.168.133.102
Then you upgrade to Intel or Broadcom NICs. Realtek NICs are acceptable for desktop computers, but if you're building an iSCSI network and you spend the money on decent switches for this iSCSI implementation, don't try to save a few dollars by using Realtek NICs (why do you think none of the server manufacturers (IBM, HP, Supermicro, Tyan, Dell, etc) use Realtek NICs on their server motherboards?).
Then you plug the connections for 192.168.130.x and 192.168.132.x into one switch/vlan and 192.168.131.x and 192.168.133.x into the 2nd switch/vlan. This is the start of a proper iSCSI setup.
rbramble
5 Posts
0
April 9th, 2011 22:00
OK,
If I set everything back to default? And replace the NIC's with other cards? How do I set up the servers? What connections do I set them up to? Any detailed doc's on this?
Thanks
Ron
JOHNADCO
847 Posts
0
April 11th, 2011 11:00
You want 4 nics in each host for iSCSI to these arrays. Each runs on a different subnet. Generally you need a dedicated switch for each subnet or at least a dedicated Vlan on a set of switches.
Powervault arrays do not do well with multiple iSCSI ports on the same controller on the same subnet.
rbramble
5 Posts
0
April 11th, 2011 11:00
Hello,
I have looked at more documentation and need to get an idea of this.
I am looking at the <ADMIN NOTE: Broken link has been removed from this post by Dell>
Appendix C, Figure 1
<ADMIN NOTE: Broken link has been removed from this post by Dell>
is this information right? Under the TCP Connections T01 shows IP_Addr_If1 / IP_Addr_C0_P0 T02, TO3, T04 shows connecting to Controller 1. However T11: through T14 shows connecting to Controller 1
Thanks
JOHNADCO
847 Posts
0
April 11th, 2011 16:00
The diagram is right for a two nic per host setup. Really, don't cheat your self on these array, use 4 nics per host. You get 4 x 1gb bandwidth.
I say that, but I must admit, I plan to only have two nics in each host for quite sum time even when we go to our new storage servers which could easily be configured for 4.
rbramble
5 Posts
0
April 13th, 2011 09:00
Hello,
I have updated my settings. It is much faster but is it as fast as the equipment can go? I have seen faster when the MPIO was not enabled but the SAN wasn't stable.
I have 2 Clusters with 2 Nodes on each Server. Each server is running Windows 2008 R2 SP1. There are 2 NIC's in each Server dedicated to iSCSI. I have 1 Switch that is dedicated to iSCSI. There are 3 VLANS on the switch, Jumbo frames Enabled, Not Tagged. All NIC's in the Servers are running Jumbo Frames 9K.All 130.xxx address are on VLAN 130, All 131.xxx are on VLAN 131. The switch and my other SAN and a NAS and the 2 Domain Controllers are also running on the iSCSI switch and their address is 192.168.135.xxx and they are all on VLAN 1.
I had a Dell 5424 Switch but lost access to the Web Management portal via the internet browser so I went back to the other single switch. I have another Dell 5424 switch that I bought coming in. Can anyone recommend how to setup the switches including access from the Web portal not the console cable?
Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4
EX-Node1 iSCSI-1 192.168.130.130
EX-Node1 iSCSI-2 192.168.131.130
EX-Node2 iSCSI-1 192.168.130.140
EX-Node2 iSCSI-2 192.168.131.140
Prod-Node1 iSCSI-1 192.168.130.10
Prod-Node1 iSCSI-2 192.168.131.10
Prod-Node2 iSCSI-1 192.168.130.20
Prod-Node2 iSCSI-2 192.168.131.20
Dell MD3200i Controller 0 Port 0 192.168.130.101 Jumbo Frames 9K
Controller 0 Port 1 192.168.131.101 Jumbo Frames 9K
Controller 0 Port 2 192.168.130.103 Jumbo Frames 9K
Controller 0 Port 3 192.168.131.103 Jumbo Frames 9K
Controller 1 Port 0 192.168.130.102 Jumbo Frames 9K
Controller 1 Port 1 192.168.131.102 Jumbo Frames 9K
Controller 1 Port 2 192.168.130.104 Jumbo Frames 9K
Controller 1 Port 3 192.168.131.104 Jumbo Frames 9K
ISCSI Settings Example. Each Server had 2 Discovery Portals, Add a MPIO Session and then 4 MCS Sessions per each TCP Target Session.
Discovery 192.168.130.101 -- 192.168.130.130
192.168.131.103 -- 192.168.131.130
Source Target
Target (1) 192.168.130.130 192.168.130.101
MCS Sessions 192.168.130.130 192.168.130.103
192.168.131.130 192.168.131.101
192.168.131.130 192.168.131.103
Target (2) 192.168.131.130 192.168.131.102
MCS Sessions 192.168.131.130 192.168.131.104
192.168.130.130 192.168.130.102
192.168.130.130 192.168.130.104
This is pretty long but it includes the settings.
Can anyone recommend an action to make it faster or reconfigure the Dell 5424 switches so they will be optimized iSCSI and connect via the Web Portal? Thanks for your assistance.
rbramble
5 Posts
0
April 15th, 2011 09:00
Hello,
I think I am done messing with this.
I used the above configuration but I also added both Dell 5424 switches. Switch #1 is running the VLAN for the 130/132 subnet and the 2nd switch is running the VLAN for the 131/133 subnet. I set up both switches identically with 3 VLAN's and only plugged in the cables into the separate ports associated with the VLAN for that switch. I have a NAS and a Vtrack SAN on the same network but put them on their own subnet.
My test was a 21 GB backup to an LTO4 tape drive. When I first started it was almost 2 hours. Last night it was 11 minutes.
Thanks for the assistance
Ron
evans_zack
2 Posts
0
May 27th, 2011 12:00
What are the default gateway and subnet mask settings for this IP settup? I'm a complete network newbie. Thank you.
dirkdigs
192 Posts
0
January 20th, 2012 20:00
what if you had a r2 cluser with 8 nic ports
would it be
node1
iscsi-1 192.168.130.x
iscsi-2 192.168.131.x
iscsi-3 192.168.132.x
iscsi-4 192.168.133.x
node2
iscsi-1 192.168.130.x
iscsi-2 192.168.131.x
iscsi-3 192.168.132.x
iscsi-4 192.168.133.x
jonnyjonah
9 Posts
0
October 3rd, 2012 08:00
Tried switching to MRU to see if this is an MPIO issue but then the writes max at 110MB/s as expected but the read is still stuck ~2MB/s
Thanks Jonathan
nid15
57 Posts
0
July 4th, 2013 13:00