Query on LUN presentation to ESX clusters

Since ESX 3.5 (i think U2) the LUN number is no longer used in identifying paths for devices. As long as the array is compliant (which these are) the WWN of the device will be used which does not change across FAs.So ESX (4/5) will not have an issue seeing these as valid paths to these devices no matter how you pair the FAs with the host initiators.

I guess i should have added a caveat to what i said. There is an issue with performing a vMotion of a VM with RDMs that do not have consistent LUN #s between the hosts and paths. It will fail until you fix it or temporarly remove the LUNs and re-add them after vMotion.

Correct. With the version of ESX indicated, as well as the Enginuity levels LUN numbers are not used in identifying valid paths. We always recommend consistent lun numbers, but since ESX changed how they identify device paths to use the WWN it is mostly just for ease of management and legacy support.

Hey Cody,

thank you for the replies..in looking at the SAN config guide for ESX 4.1 it states that LUN addressing still has impact with multipathing? 

>>

Fibre Channel SAN Configuration Guide   ESX 4.1

For multipathing to work properly, each LUN must present the same LUN ID number to all ESX/ESXi

hosts.

Storage provisioning

To ensure that the ESX/ESXi system recognizes the LUNs at startup time,

provision all LUNs to the appropriate HBAs before you connect the SAN to the

ESX/ESXi system.

VMware recommends that you provision all LUNs to all ESX/ESXi HBAs at the

same time. HBA failover works only if all HBAs see the same LUNs.

For LUNs that will be shared among multiple hosts, make sure that LUN IDs

are consistent across all hosts. For example, LUN 5 should be mapped to host

1, host 2, and host 3 as LUN 5.

Okay, based on some of this discussion, stumbled across this KB which appears to indicated 3.5 U5 and up does not rely on LUN ID, but NAA support (Network Address Authority)...NOTE that SYMMETRIX supports NAA usage as long as the SPC2 bit is set (as documented).

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1026710

>>

Network Address Authority (NAA)  vs. LUN IDs

This behavior is prevented through the support and use of NAA for SAN LUNs with VMware ESX 3.5 Update 5 and later.  We did introduce the supportability for NAA ID's with ESX 3.5 GA, however we stopped using the LUN ID to reference the device from Update 5 onwards.

If the SAN array or LUNs do not support NAA, the SAN presentation of LUN IDs must be uniform or consistent across all hosts/initiators. Thus versions of VMware ESX prior to 3.5 Update 5 require uniform LUN IDs and storage presentation. See VMFS resignaturing in the Additional Information section for more information.

>>

any additional comments or suggestions from anyone?

thank you

So, for multiple ESX nodes prior to 3.5 Update 5, Uniform LUN ID is required.  Is this LUN ID the one mapped on symmetrix FA which can be designated by storage admin from symmwin?

from symcli it could be either the address on the FA or if you are using dynamic lun masking it's the available LUN id, not necessarily address on the FA.

Hi Cody,

Could you please comment on how other host OSs\multipathing software identify LUNs - eg, Veritas DMP, windows, Linux, etc. Do they need to see the same LUN ID after being rebooted or if LUNs are removed & re-added?

I am not to worried about clustered servers, my concern is for a DR environment where I have to remove the R2s & add gold clones to the DR hosts as the need be. I am planning to to use nested SGs so the LUN ID may change when I remove the R2s & add the Gold clones.

Gary

I am not really sure how other OSes identify them, but my understanding is that most are more tolerant when it comes to LUN IDs. LUN 0 can be an issue for some though. If you have specific OS in mind I would try posting in those communities on ECN (Everything Microsoft etc.).

Most modern systems (Linux DM, Windows 2008+, modern DMP, PowerPath) use the device WWN or NAA id to identify a device, and so LUN ID generally doesn't matter.  Now, that means you need to pay attention to which devices you are putting filesystems on, for example.

Example:  Using device mapper on linux, a given LUN can be accessed via:

/dev/sdX, Y and Z (the various individual paths, not multipathed and not failover safe)

/dev/mapper/dm-XXX (a multipathed device, but one that could be a different device if you add LUNs with lower LUN IDs, because they are enumerated in order

/dev/mapper/by-id/YYYYYYYYYY (a multipathed device that ALWAYS corresponds to the same device, regardless of LUN ID.  If the device changes LUN ID's, no change to this block device.  If LUNs are added or remove, no change.  This is a safest option.

Now, most applications that use raw disks or have modern filesystems can handle a LUN ID change anyways, but its still best to be aware.

Vsphere 5.5 never requires LUN ID's to be the same.

Thanks for the reply Cody. I will collect a list of OSs & multipathing software & validate them. Thenak for your help.

Nice work on the blog - I am following it closely.

Thank you Matt. This helps.