SPA can see luns but SPB cannot.

Question

I have a ClARiiON CX3-20 with 2 Cisco mds 9000 series switches. I am running VMware ESX version 3.0.2 with 4 host servers. The host servers each have 2 hba cards, one connected to each switch. The problem is all my ESX host servers can see all luns only through SPA. Nothing is showing up from SPB on any host. Secondly, no luns are showing up on the hba card connected to the second switch, in 3 out of 4 servers. You would think the ESX servers would at least be able to see the luns on the 2nd hba card and switch through SPA? There are no errors or alerts of any kind in Navisphere for SPB. I replaced all fiber cables connected to SPB. The Cisco switches are setup so each server is in it's own zone. When ever I manually treaspass the luns to SPB they move back to SPA. My thoughts are, I am most likely looking at multiple issues. First, there must be a problem with SPB that is not being reported by Navisphere, Frabric manager or VCenter. Secondly, there must also be a problem with the hba cards. The hba cards are supposed to be Qlogic 2460 Es. I say supposed to be because the cards have a chip that says 2422, which is what VCenter says they are. The hba cards also have a sticker that says 2460. So which one are they? Does anyone have any thoughts or have had simular issues?

Thanks.

RRR · Accepted Answer

Ah, right... I forgot. In that case ESX should be able to see all 4 paths to each lun. So even is a lun is default on SPA and is currently not tresspassed, ESX should see the path to the lun using SPB as well.
I'm completely lost now.

If the zoning is correct, port speeds are set correctly and in Navisphere all 4 paths are active (check connectivity status or - in engineering mode - check the hosts / advanced tab in the VMware-storage group.... if all this checks out, I guess it should work just fine.

RRR · Answer

Is each SP connected to switch1 as well as switch2 ?Is each HBA connected to a port on SPA as well as a port on SPB by proper zoning ? So each host should have 4 zones or at least 4 paths to each lun.

nandas · Answer

Is this a new Install or existing one - I mean, were everything working properly before and now is a problem? If the setup is in production, better to engage proper Support teams at the earliest.

Have you checked the Switch connections and especially the zoning on each switch for all the servers? As Rob mentioned, each host must have 4 paths to each LUN - 2 on one switch and 2 on the second using both the HBAs connected to each switch.

Hope your problem gets resolved quickly.

Thanks,
Sandip

kelleg · Answer

I would say that you need to verify some configuration steps first - also, check on PowerLink - there are Host Connectivity guides for all operating systems:

1. Is each HBA zoned to an SPA port and an SPB port? Can you see this in the active zone on the switch? Is the active zone saved and active? Try activating the zone again. Are the servers logged into the array - check in the Connectivity Status page on the Array- right click array and select Connectivity Status. Make sure all paths are logged in, if you're missing connections, check your zoning.

2. Are the LUNs on the array owned by SPA or SPB? What is the failover setting for each host - VMware requires you set have failover set to 1 (one) for each path.

3. Are the servers correctly configured in the Storage Groups? Are icons for all paths in Connectivity Status Green or Blue? Green means logged in, registered and connected to a storage group. Blue means you're missing the storage group.

4. Trespassing LUNs using the array is not considered best practice as the server will sense that the LUN has trespassed and will trespass the LUN back to where it last was. You should change the default owner of the LUN first, then use the server failover software to trespass the LUN.

5. On the Qlogic HBA - the chip, 2422 is a family name - the 2460 uses the same chip - go by the sticker on the card. Also, make sure that you obtain the HBA drivers from the EMC section of Qlogic WEB page.

regards,

glen kelley

wishtobeMrEMC · Answer

As you may aware, technically a lun can be owned only by one SP at a time, not by both. please ensure luns are owned only by SPA in your case. you may want to change the ownership of the lun to SPB and try.

RRR · Answer

4. Trespassing LUNs using the array is not considered best practice as the server will sense that the LUN has trespassed and will trespass the LUN back to where it last was. You should change the default owner of the LUN first, then use the server failover software to trespass the LUN.

In my expierience I saw something else happening when a manual tresspass is invoked: the attached servers which are using that lun will follow the lun to it's new SP and start using a different path. However: other servers which weren't using that lun will still have the old path as preferred path. The MRU setting on those servers will perform a tresspass back to the original path (SP) as soon as that lun is used by any one of the other servers.

JGREUNKE · Answer

RRR that is correct, each switch is connected to SPA & SPB and every server has 2 hba cards one connected to each switch.

JGREUNKE · Answer

Thanks for your input kelleg. I have Linux ESX servers so they don't use power path. I've been told the Linux OS takes care of failover. The failover for each path is set to one. All servers are in storage groups and the connectivity status says everything is registered and logged in. What you said about the manual treaspass makes sense. The luns are divided up between the two SPs for ownership. However, I restarted the CX3-20 and the luns all went over to SPA. So what is causing the luns to move to SPA?. Could this be the Linux OS on the ESX servers causing this or is there a weird problem going on with the SP itself? Good info on the HBA cards:D

Thanks.

RRR · Answer

The only reason I can think of why all luns move to SPA is that their default owner is SPA. A best practice it to evenly devide luns over both SPs, let's say all odd luns to SPA and all even luns to SPB.

And in answer to the failover thingy of Linux: why not use Powerpath ? You can use the unlicensed version, which does exactly that !

RRR · Answer

It seems all luns are on SPA and none on SPB. It could be that simpel.

If you enter Navisphere and go to "tools" in the pull down menus, choose "tresspassed luns status". Are there any luns there ?

If you open the "lun folders", what do you see underneath SPA and SPB ? Are there any luns under SPB ?

If you don't have any luns under SPB and there are no tresspassed luns, every lun has SPA as default owner ! You can manually edit the luns you'd want under SPB and make SPB the default owner of those luns. When you changed that, check tresspassed luns again and you'll see that every chenged lun is now tresspassed. You can manually tresspass them back to their default owner.
Now you should see luns on SPA as well as on SPB on the servers.

JGREUNKE · Answer

This is a new install of the second switch and second HBAs, in the host servers. The CX3-20 was existing. I wanted to eliminate as many single points of failure as possible by adding a second switch and the HBA cards. The original setup with one switch and 1 HBA per host server worked fine to the best of my knowledge. I did not setup the SAN, a local consultant did, so I am not sure if SPB was working back then or not. One out of four of my host servers can see all luns on both HBA cards, but only through SPA. I actually had to replace both HBA cards in that server. The original cards were Qlogic 2340s. I replaced them with Qlogic 2460 Es. I could not get the 2340s to work at all, after adding the second switch. There should be two Qlogic 2460 Es in all host servers.

Thank You for your response.

AranH1 · Answer

And in answer to the failover thingy of Linux: why
not use Powerpath ? You can use the unlicensed
version, which does exactly that !

He is running ESX Server which doesn't yet support Powerpath even though it runs on a Linux kernel. VMWare ESX server has it's own failover method.

RRR · Answer

Did you create a service request ? I think it's time to do so now. It seems nobody knows how to handle this problem you're seeing.....

AranH1 · Answer

I'm completely lost now.I know I have read over this post and it just seems like something is missing.

kelleg · Answer

Check the failover mode for the ESX servers - VMware uses MRU - you need to set this for each LUN from the host.

On VMware failover, when ESX senses that it's lost connection to the LUN down one path, it will trespass the LUN to the other path. Once the other path is restored, ESX will NOT trespass the LUN back to the default owner.

Check the default owner for each LUN and see if they are evenly divided between SPA and SPB, not the current owner,

When you reboot an array, one SP may go down quicker than the other and the host might try to trespass the LUNs when it senses that the SP path is gone.

glen

CLARiiON

SPA can see luns but SPB cannot.

Was this post helpful?