Unsolved
This post is more than 5 years old
14 Posts
0
29252
October 13th, 2008 06:00
MD3000i and vmware : redundant iscsi setup
Hi All,
I'm struggling with this for several weeks now, and hope to find some usefull advice in here.
This is our equipment :
- MD3000i with dual storage processor
- 2 x Powerconnect 2716 switches
- 2 hosts with 3x dual Gigabit Nic (of which 2 (out of 6 connections) are dedicated used for iscsi)
- VMWare esx 3.5 with software iscsi initiator on the Host (not on the vm guests)
I followed the instructions as found at :
http://www.dell.com/downloads/global/solutions/esx_storage_deployment_guide_v1.pdf
And also the following discussion at this site helped me a lot :
http://www.delltechcenter.com/thread/1267618/MD3000i+and+ISCSI+Network+IP+Ranges
Here 's what I discovered untill now :
1) The one-subnet setup doesn't work that good !
- in VM : one virtual switch (with service console and vmkernel) for iscsi. Two physical NIC's attached in NIC Teaming (default settings)
- Since my switches do not support "Stacking" or "ISL" I first decided not to connect them to each other.
As a result :
- only 2 of the 4 available network-paths were 'alive'
- if the active nic or active switch went down, no failover to nic 2 was done ... until a manual rescan of the storage adapters was initiated
Connecting the two switches (with a standard crosscable) solved this "a bit", but :
- all connections were initiated by the first nic (no load balancing over the 2 nics)
- more often than not, the most NON-optimal 'data paths' were used. I mean : [nic1-switch1-switch2-san] while it could have been [nic1-switch1-san]
- And worst of all : in case of a failure of the interswitch-link (which can't be monitored) a subsequent failure of nic1 or switch1 would cause a failure of the data-connection (and a manual rescan to fail-over to nic2)
continued in next post ...
I'm struggling with this for several weeks now, and hope to find some usefull advice in here.
This is our equipment :
- MD3000i with dual storage processor
- 2 x Powerconnect 2716 switches
- 2 hosts with 3x dual Gigabit Nic (of which 2 (out of 6 connections) are dedicated used for iscsi)
- VMWare esx 3.5 with software iscsi initiator on the Host (not on the vm guests)
I followed the instructions as found at :
http://www.dell.com/downloads/global/solutions/esx_storage_deployment_guide_v1.pdf
And also the following discussion at this site helped me a lot :
http://www.delltechcenter.com/thread/1267618/MD3000i+and+ISCSI+Network+IP+Ranges
Here 's what I discovered untill now :
1) The one-subnet setup doesn't work that good !
- in VM : one virtual switch (with service console and vmkernel) for iscsi. Two physical NIC's attached in NIC Teaming (default settings)
- Since my switches do not support "Stacking" or "ISL" I first decided not to connect them to each other.
As a result :
- only 2 of the 4 available network-paths were 'alive'
- if the active nic or active switch went down, no failover to nic 2 was done ... until a manual rescan of the storage adapters was initiated
Connecting the two switches (with a standard crosscable) solved this "a bit", but :
- all connections were initiated by the first nic (no load balancing over the 2 nics)
- more often than not, the most NON-optimal 'data paths' were used. I mean : [nic1-switch1-switch2-san] while it could have been [nic1-switch1-san]
- And worst of all : in case of a failure of the interswitch-link (which can't be monitored) a subsequent failure of nic1 or switch1 would cause a failure of the data-connection (and a manual rescan to fail-over to nic2)
continued in next post ...
No Events found!
koen.vdvelde
14 Posts
0
December 2nd, 2008 06:00
I appreciate your reply on this post and 'll try to answer your questions as good as I can :
1 & 2) In network config, I did create 2 VSwitches for ISCSI, each with its own Service Console AND VMKernel. However, in Storage Adapters, there's only one iscsi initiator. In it's 'dynamic discovery'-tab I added the 4 ip-adresses (2 in each network range) of my san.
port 0-0 : 10.102.1.100
port 0-1 : 10.102.2.101
port 1-0 : 10.102.2.110
port 1-1 : 10.102.1.111
There's only one iscsi Software adapter (vmhba32) (and only one iQN-number) under storage. I never added this one myself, it was there after enabling 'Software iSCSI Client' under Security profile and a rescan under storage.
3) If (and only if ;-) I remember well .. I hit rescan only once
4) I'm not sure how multipathing is working. On thing is sure : in this setup there is no NIC Teaming configured (since there's only one nic attached to each of the VSWitches).
The iscsi initiater knows the 4 possible IP-targets of the SAN and is able to connect to each of them (simultatious). Which one of this 4 available paths will be used, is decided automaticly. However, it can be fine-tuned on a per-LUN basis, by selecting a "fixed/prefered" path in the "Manage Paths"-dialog box.
... to be continued ...
tonyhgras
9 Posts
0
December 4th, 2008 15:00
Thanks for your response. I've been busy and didn't have time to post a reply.
Following the iSCSI configuration in VMware discussion, here are a few links I want to share with you (and the other thread contributors).
First, the PowerConnect 2716 User's Guide Addendum (please find information about LAG configuration on page 5):
http://support.dell.com/support/edocs/network/PC27xx/en/ug/addug/appa.pdf
This one is an EXCELENT article about NIC teaming in ESX, with valuable tips for IP Storage. As you surely know, when you use the one-subnet iSCSI setup, NIC teaming is the means for achieving redundancy to storage, and the most one can learn about teaming configuration, the best one can configure the iSCSI network. Here's the link:
http://blog.scottlowe.org/2008/07/16/understanding-nic-utilization-in-vmware-esx/
Here's another "gem" for enabling "jumbo frames" for iSCSI traffic on ESX 3.5 (although not "officially supported", it should be experimental only):
http://blog.scottlowe.org/?s=ESX+Server%2C+IP+Storage%2C+and+Jumbo+Frames
And finally, the confirmation that VMware "officially support" the two-subnets iSCSI setup (please check by yourself at page 207 of the following document):
http://www.vmware.com/pdf/vi3_san_design_deploy.pdf
From the last docs and articles I've read about iSCSI (software based) on ESX, the dual-subnet setup seems to me the preferred option. I find it simpler to configure, less dependent on networking equipment features, and more robust regarding multipathing and (elementary) load balancing. I would like to know what do you think, if you have time to share your thoughts.
Best Regards,
Tony
planetelex
2 Posts
0
January 18th, 2009 13:00
Does anyone here know whether I need to use a LACP or LAG configuration on the trunked uplink port if I am using a single subnet configuration. I don't have any dell switch experience. I have dell 5424 switches and I have read the manual and can see that I need to configure a trunk between the 2 x uplink ports for the single subnet config but would appreciate it if someone can advise on the correct LAG or LACP commands or provide a example 5424 configuation.
Thanks
tonyhgras
9 Posts
0
January 19th, 2009 09:00
Assuming that when you said "trunk" you mean "team" when refering to the ESX physical uplink ports dedicated to iSCSI, and that this ports are connected to two different physical switches (PowerConnect 5424) for redundancy, I would say that you DON'T need to configure LAG on the external switches for this setup for the ports facing the ESX host.
Indeed, I think you CAN NOT do this between ports in two different physical switches, unless those switches are stacked together, which is not possible with a pair of PC 5424's.
For this model of switches the only way you have for connecting the two switches together is by means of ISL's (eg: linking one or more ports from each switch), since they don't support stacking. If you choose to link two or more ports between switches for this setup (highly recommended), then you can define a LAG group for this group of ports to aggregate traffic and for redundancy purpose. The bottom line is that you will need LAG only for inter-switch traffic, and not for ESX-to-Switches iSCSI traffic.
Using this one-subnet setup with non-stackable switches, you can only get failover, but not Load Balancing for ESX iSCSI traffic. Too bad, I think.
If you want to go deeper on this "ESX networking features" related to iSCSI you can check (besides VMware docs, of course) this superb article from a popular VMware blogger:
http://blog.scottlowe.org/2008/07/16/understanding-nic-utilization-in-vmware-esx/
Hope this helps you. Bests regards,
Tony
Phuzzi
13 Posts
0
January 23rd, 2009 01:00