Problem with Host iSCSI initiators and SP ports on same subnet

A port reset on the array could be caused by a number of things - connectivity issues on the port, lose cable, etc. See emc245445 for a list of all known iSCSI issues. There also may be an issue on the port itself. Some of the articles are for array port issues.

glen

Thanks Glen for the reply.

I checked with the citrix server team and the networking team...even though all the server NICs and all the SP iSCSI ports are on the same subnet...only one NIC/server is active other one being passive.

The way I see this, its either

1. The networking/server side issue which caused two NICs from the same server try to login to same SP iSCSI port which might have caused the IO module to reset. BUT, since the server team tells me that only one NIC/server is active so I am guessing this was not the case.

or

2. The IO module is failing? (I don't want to jump to this conclusion because this is the first time something like this has happened)

Please correct me if I am wrong

also, is host side NIC bonding supported on Clariion? (CX4-120)

NIC Teaming is not supported with the MS iSCSI Initiator; Microsoft doesn't support it so of course we don't either.  This is mentioned in the E-Lab Interoperability Navigator output when you first qualify your environment:

https://elabnavigator.emc.com/do/navigator.jsp

For example:

Also the "EMC Host Connectivity Guide for Windows" repeats the same and is available from PowerLink via the following breadcrumb trail:

Home > Support > Technical Documentation and Advisories > Host Connectivity/HBAs > Installation/Configuration

Active or not, we do not support more than one initiator on the same host logging into the same target.  This is mentioned scattered throughout our KB articles and documentation.  To control this behavior that is also why you are expected to explicitly pair up the intiators and targets (using "Advanced Settings" in the MS iSCSI initiator when logging into the targets and not accepting "Default" for "Source IP" and "Target Portal").  This is demonstrated in the EMC Host Connectivity Guide for Windows and several of our KB articles.  Separate isolated subnets (recommend not setting up a default gw if possible) will prevent this of course, and is just one of several reasons why we require separate subnets as you rightfully identified.

Just one more point, it is not only important to separate your iSCSI subnets from each other but also from the array's (SP and CS) mgmt IP's.  We've actually introduced a Navisphere Alert that will warn when an iSCSI interface on the SP's shares the same subnet as the SP mgmt ports as a reminder.  Finally, since they are internal subnets used by the CLARiiON (and Celerra), it is also important that none of the subnets you define use: 192.168.1.0/24 or 128.221.0.0/16.

As Glenn pointed out though, review the KB article he referenced as it includes these points along with many other iSCSI configuration best practices.

Oops, sorry just reread and realized that I referenced Microsoft in regards to the NIC Teaming but you were running Citrix Xenserver.  Support is generally per the OS vendor.  For instance, to put it into perspective, VMware ESX/ESXi supports NIC Teaming of the vSwitch (which includes the VMkernel Ports) while Microsoft doesn't as mentioned above.  Similiarly as with the Microsoft support statements, VMware's stance can be found in the "EMC Host Connectivity Guide for VMware ESX Server".

Not completely familiar with Citrix Xenserver myself, maybe someone else can comment from experience; however, again, generally support is per the OS vendor.

What you want is to ensure that each NIC on each host is only configured to log into one SP port per SP. In Windows you can specify the Source and Dest. IP addresses in the iSCSI Initiator Control Panel. Not sure how you can do this on Citrix.

glen

Nobody would use 128.221.0.0/16 as it is Host name: www.cs.isus.emc.com

But to define 192.168.1.0/24 (most popular private network) is just MAD!

Could you not make it something LESS likely to be used ie 192.168.177.0 ?

Seb

Check the Flare code I suspect its a bug

Thank you everyone for your input, as I said before the configuration was done (before I joined at the client) with all the ports on same subnet.

All the iSCSI ports on clariion are configured as 192.168.199.10-13.

All the Citrix servers have IPs 192.168.199.90-100

Flare code version is 30 (4.30.000.5.517)

I am working with networking team to make sure the configuration is changed according to EMC best practices of defining 2 subnets with 1 NIC/iSCSI HBA and 1port/SP per subnet.

So I finally had the networking team/server team and myself configure the ports according to EMC's recommended best practices (SPA6 and SPB6 on subnet 1 and SPA7 and SPB7 on subnet2).

Things look good from both server and storage sides, but I had a question about the total paths and sessions.

So before when all the NICs/SP ports were on a single subnet (NIC bonding on host side was being done which is not the case anymore) we saw 4 paths and 1 active session.

After the config change we were hoping to see 4 paths (since no more NIC bonding on server side) and 2 sessions BUT we only see 1 session again (with 4 paths).

So if someone can explain why this is happening, I would really appreciate it. I am trying to understand if this is something from the array side.

All NIC's in the same host have the same IQN name (different IP addresses) - this is used to establish the session with the array. If you have two host NICs with two connections per NIC, you have a total of 4 connections using the same IQN name.

You failover software determines what paths are active and passive (or standby). PowerPath will use both paths to a LUN and load balance between the two NIC (each NIC has a path to SPA and SPB - if the LUN is owned by SPA< then you have one path to SPA from each NIC - two paths total). Other operating systems use a different method, ie. ESX as a Fixed path policy that will send IO down only one path and the others are standby. The Round Robin polidy will sort of balance the IO down the two paths to a LUNs - alternating between one and the other.

glen

ALUA allows IO to be processed by both SPs for the same LUN; however there is a performance impact to IO that is processed by the non-owner SP. As such, under normal circumstances, ALUA compliant hosts will direct traffic to the owning SP only. If you have 2 paths to each SP, then the answer to your question is yes, only 2 paths will handle IO for a specific LUN at a time. Ensure that LUNs are evenly distributed between the two SPs though and you’ll see traffic on all 4 ports.

Richard J Anderson

correct, a LUN is owned by one SP at a time so there will be two active paths to the current SP.

The Citrix team got a pdf from their support (http://support.citrix.com/article/CTX126121) which specifies how to configure their servers to access storage from clarrion.

Following this, using ALUA failover mode, we can see 4 paths and 4 sessions.

Citrix has built in Round Robin policy and Powerpath doesn't support Citrix Xenservers.

So if a lun is owned by SPA does that mean I have only 2 paths actually being used and not all the 4 paths? (even though it shows up as 4 active paths)