This post is more than 5 years old
7 Posts
0
8764
MPxIO and Clariion CX4-480
Hi
We have encountered a pathing issue with Solaris MPxIO and a clariion CX4-480. Here are some details regarding our config:
CX4-480
Initiator type: CLARiiON Open
HBA type: Host
Array CommPath: Enabled
Failover Mode: 1
Unit Serial: Array
Connection
Initiator SP Type Logged Registered
1 B-0 Fibre Yes Yes
2 A-3 Fibre Yes Yes
3 A-0 Fibre Yes Yes
4 B-3 Fibre Yes Yes
Luns
Name ID Capacity Type Hostid
hostname /oraredo 47 40.000 GB FC 0
hostname /oracrm 80 50.000 GB FC 1
hostname /orabiz 81 50.000 GB FC 2
hostname /orasoft 92 100.000 GB SATAII 3
hostname /oraarch 97 150.000 GB FC 4
hostname /orafra 96 200.000 GB FC 5
Format output from solaris
Searching for disks...
The current rpm value 0 is invalid, adjusting it to 3600
The current rpm value 0 is invalid, adjusting it to 3600
done
c1t5006016B3CE00FACd0: configured with capacity of 40.00GB
c1t500601603CE00FACd0: configured with capacity of 39.98GB
c2t500601633CE00FACd0: configured with capacity of 39.98GB
c2t500601683CE00FACd0: configured with capacity of 40.00GB
c3t60060160294122002A2ADDB296EADE11d0: configured with capacity of 150.00GB
c3t6006016029412200C28485A496EADE11d0: configured with capacity of 199.99GB
c3t60060160294122003CD924D692EADE11d0: configured with capacity of 50.00GB
c3t6006016029412200D219342394EADE11d0: configured with capacity of 100.00GB
c3t60060160294122003DD924D692EADE11d0: configured with capacity of 50.00GB
AVAILABLE DISK SELECTIONS:
0. c1t5006016B3CE00FACd0
/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/ssd@w5006016b3ce00fac,0
1. c1t500601603CE00FACd0
/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/ssd@w500601603ce00fac,0
2. c2t500601633CE00FACd0
/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/ssd@w500601633ce00fac,0
3. c2t500601683CE00FACd0
/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/ssd@w500601683ce00fac,0
4. c3t5000C5000A6B4BF7d0
/scsi_vhci/disk@g5000c5000a6b4bf7
5. c3t5000C5000A6B4D97d0
/scsi_vhci/disk@g5000c5000a6b4d97
6. c3t60060160294122002A2ADDB296EADE11d0
/scsi_vhci/ssd@g60060160294122002a2addb296eade11
7. c3t6006016029412200C28485A496EADE11d0
/scsi_vhci/ssd@g6006016029412200c28485a496eade11
8. c3t60060160294122003CD924D692EADE11d0
/scsi_vhci/ssd@g60060160294122003cd924d692eade11
9. c3t6006016029412200D219342394EADE11d0
/scsi_vhci/ssd@g6006016029412200d219342394eade11
10. c3t60060160294122003DD924D692EADE11d0
/scsi_vhci/ssd@g60060160294122003dd924d692eade11
luxadm
Found Fibre Channel device(s):
Node WWN:50060160bce00fac Device Type:Disk device
Logical Path:/dev/rdsk/c1t5006016B3CE00FACd0s2
Physical Path:
/devices/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/ssd@w5006016b3ce00fac,0:c,raw
Logical Path:/dev/rdsk/c1t500601603CE00FACd0s2
Physical Path:
/devices/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0/fp@0,0/ssd@w500601603ce00fac,0:c,raw
Logical Path:/dev/rdsk/c2t500601633CE00FACd0s2
Physical Path:
/devices/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/ssd@w500601633ce00fac,0:c,raw
Logical Path:/dev/rdsk/c2t500601683CE00FACd0s2
Physical Path:
/devices/pci@0,600000/pci@0/pci@9/SUNW,emlxs@0,1/fp@0,0/ssd@w500601683ce00fac,0:c,raw
Node WWN:50060160bce00fac Device Type:Disk device
Logical Path:/dev/rdsk/c3t60060160294122002A2ADDB296EADE11d0s2
Physical Path:
/devices/scsi_vhci/ssd@g60060160294122002a2addb296eade11:c,raw
Node WWN:50060160bce00fac Device Type:Disk device
Logical Path:/dev/rdsk/c3t6006016029412200C28485A496EADE11d0s2
Physical Path:
/devices/scsi_vhci/ssd@g6006016029412200c28485a496eade11:c,raw
Node WWN:50060160bce00fac Device Type:Disk device
Logical Path:/dev/rdsk/c3t60060160294122003CD924D692EADE11d0s2
Physical Path:
/devices/scsi_vhci/ssd@g60060160294122003cd924d692eade11:c,raw
Node WWN:50060160bce00fac Device Type:Disk device
Logical Path:/dev/rdsk/c3t6006016029412200D219342394EADE11d0s2
Physical Path:
/devices/scsi_vhci/ssd@g6006016029412200d219342394eade11:c,raw
Node WWN:50060160bce00fac Device Type:Disk device
Logical Path:/dev/rdsk/c3t60060160294122003DD924D692EADE11d0s2
Physical Path:
/devices/scsi_vhci/ssd@g60060160294122003dd924d692eade11:c,raw
As can be seen for the lun presented as hostid 0 from the array mpxio has not created the logical path and format detects 2 different sizes for the lun (suspect this is something to do with the active/passive connection)
I have been searching for any similar issues but can't seem to find a fix for this that doesn't involve a reconfig reboot. Can someone advise if there is an issue with the lun presented as hostid 0 (is this a reserved lun of some sort?)
Thanks
Adam
DavidBeatty
2 Posts
1
February 29th, 2012 06:00
I've found a solution for CLARiiON/VNX to correctly configure new paths for Solaris MPxIO and new paths for host LUN id 0. First, run cfgadm -al | grep '::5006', this should show the connection points for CLARiiON/VNX storage devices (since their WWPNs all begin with 5006); note for Symmetrix it's 5000. Make a note of the newly added connection points. For each of these, run cfgadm -c unconfigure c#::WWPN, e.g. cfgadm -c unconfigure c9::5006016944600811. Once the newly added connection points are unconfigured, run devfsadm -C to cleanup the device table. Now for the newly added connection points, run cfgadm -c configure c#::WWPN. Finally, once the newly added connection points are configured, run devfsadm. If you now run mpathadm list lu, the device for host LUN id 0 should now have the correct number of paths.
kelleg
4.5K Posts
0
April 8th, 2010 13:00
In the Host Connectivity Guide on page 147 is talks about setting up using MPxIO
Home >Support > Technical Documentation and Advisories >Host Connectivity/HBAs > Installation/Configuration
see the EMC Host Connectivity Guide for Sun Solaris -- A24
http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Technical_Documentation/300-000-607.pdf
glen
kelleg
4.5K Posts
0
April 8th, 2010 14:00
Not being a server guy, I was once told by a Solaris support person that after all configuration changes involving Solaris that it's a good practice to do something called reboot/reconfig.
Sort of like with NT - reboot for all issues
glen
aprince1
7 Posts
0
April 8th, 2010 14:00
Hi Glen, thanks for the link, I have already read this document and confirmed our solaris config is correct. We also opened a support call with Sun who confirmed this as well.
Other luns presented from the array are pathed correctly via mpxio i.e a psuedo lun is created and 4 paths are visible from `mpathadm list lu` but not in format. The issue is with this individual lun, we see all 4 paths in format rather than the pseudo path which hasn't been created and format also shows the luns with different sizes (see the cylinders in the format output and the geometry error)
Has anyone come across this error before?
aprince1
7 Posts
0
April 8th, 2010 15:00
Glen
I am predominantly a solaris admin and have minimal experience with Clariion's which is my reason for coming on these forums. mpxio is enabled on the solaris server with stmsboot -e (or varying options depending on how you want to configure mpxio) as part of that process a reboot is required to implement the changes.
After the reboot the luns were assigned to the server, mpxio automatically creates the pseudo lun for 5 out of the 6 luns presented. It is just the first one we have issues with (hostid 0) and we are struggling to identify whether this is a config problem on the server or with the array / lun. Hopefully this is something we can resolve without having to reboot all the time
d2dowling
30 Posts
0
April 8th, 2010 16:00
A couple of comments & questions:
Was lun 47, host-id 0 previously attached to another server or was it 'freshly' bound for this system?
I doubt if it is causing your ptoblem, but using naviseccli would allow you to present lun 47 to the server with a different host-id.
What version of Solaris is in use? Failovermode 4 may work better.
Excerpt from emc99467:
For Solaris MPxIO or STMS, failovermode settings are as follows:
• FLARE 24 or earlier: failovermode MUST be set to 1
• FLARE 26 or later:
- Solaris 9 with SAN 4.4.7 thru 4.4.11: failovermode MUST be set to 1
- Solaris 9 with SAN 4.4.12 or later: failovermode may be set to 1 for
active/passive behavior, or
- Solaris 9 with SAN 4.4.12 or later: failovermode may be set to 4 for ALUA
behavior
- Solaris 10 U2 or earlier: failovermode MUST be set to 1
- Solaris 10 U3 or later: failovermode may be set to 1 for active/passive
behavior, or
- Solaris 10 U3 or later: failovermode may be set to 4 for ALUA behavior
Note: Use of failovermode = 4 (ALUA) with MPxIO/STMS cannot be supported
with clustering environments requiring SCSI2 Reservations such as 2 node Sun
Clusters.
aprince1
7 Posts
0
April 8th, 2010 18:00
This server is running Solaris 10u8 and the array is at flare 28.6 I believe. I'm not entirely sure whether the lun has been used before, i'll get clarification on that. How would that impact the pathing of the lun if it had been used previously ?
I'll look into the suggestion of failover mode 4 but it seems odd that all the other luns are fine with mode 1.
Has anyone come across this specific issue before with solaris and mpxio? Have any of the suggested changes above worked.
Thanks for all the comments so far, much appreciated.
kelleg
4.5K Posts
0
April 9th, 2010 10:00
I've attached the SUN E-Lab PDF - there are a lot of notes about connecting using MPxIO - there might be something here that will help - look for the section on Clariion and find your model/OS version and match to the clariion model.
glen
1 Attachment
Sun_Solaris.pdf
StuartA461
63 Posts
0
April 9th, 2010 11:00
Why don't you use PowerPath? Is that an option?
Stuart
SKT2
1.3K Posts
0
April 10th, 2010 03:00
Is your multipath.conf tuned to the best practice for EMC clar array?Such as vendor specific "devices" section for example
aprince1
7 Posts
0
April 11th, 2010 08:00
Glen - thanks for that doc I hadn't seen that one before and it has highlighted a couple of patches that are down rev. Looking at the details for each of these patches mpxio was not a bug fix in the release. I will point these out to Sun and EMC and see if we can get some clarification on whether they are causing our issue.
Stuart - powerpath isn't really an option, all other solaris servers are using mpxio and working ok at present and this is the standard setup for the environment.
SKT - there is no multipath.conf on solaris, I believe that is linux related. The config files relating to mpxio have been configured as per the following docs:
host connectivity guide & stms for clariion.
Thanks for your help so far
Adam
kelleg
4.5K Posts
0
April 12th, 2010 12:00
Adam,
Do you have a case open with EMC at this point?
glen
aprince1
7 Posts
0
April 12th, 2010 13:00
Glen
I've had a call open with Sun for about a week or so now and I raised one with EMC on Friday last week. I'll keep you updated with the outcome
Cheers
Adam
aprince1
7 Posts
0
April 24th, 2010 10:00
Hi Guys
Just a bit of an update for you, neither EMC or Sun have been able to provide an explanation as to what has caused this issue or how to correctly rectify it. EMC's suggestion was to remove the lun from the storage group and reattach it again, whilst this did sort the pathing problem and create a new pseudo lun we are left with entries still showing in format, luxadm, cfgadm etc.
These have been tidied with the following:
luxadm -e offline (disk path)
devfsadm -C -c disk
cfgadm -c unconfigure -o unusable_SCSI_LUN cX
When the host is rebooted the removed lun re-appears with "drive type unknown" in format as if the lun is still presented to the server (the same happens after a reconfig reboot as well). I'll let you know of any further updates
Cheers
Adam
kelleg
4.5K Posts
0
April 26th, 2010 18:00
When you look at the Storage Group Properties that contains this host, look at the LUNs tab (right-click on Storage Group name and select Properties). Each LUN in the storage group will have an Array LUN Number (ALU) and a Host LUN Number (HLU) - do you have one LUN that is HLU 0 (zero)? If not, you need to add a small LUN (1MB) to the Storage Group and assign it HLU 0 - just click in the column where HLU is before you hit apply.
glen