Announcement Banner
UNSOLVED

gedd1

updated

16 years ago

G

gedd1

9 Posts

0

34009

December 29th, 2010 12:00

MD3200, Solaris 10 multipath

Hi there :)
I'm trying to configure Solaris 10 (on Dell PE R710) to work with MD3200 via H200 SAS Controller.
Server is connected with Storage Array with two cables:
- port 0 of HBA to port 0 of first MD3200 controller
- port 1 of HBA to port 0 of second MD3200 controller

As expected iostat in Solaris shows every LUN twice.
Anyone knows how to do multipath in this case in Solaris 10 (update 9)?

Thanks in advance !
  • mrokkam1

    23 Posts

    1809

    0

    Posted December 29th, 2010 13:00

    Dell does not officially support solaris with the MD3200. LSI's 2600 systems are the more or less the same and you can attempt to get their Solaris RDAC drivers...
    -Mohan
  • gedd1

    9 Posts

    1809

    0

    Posted December 29th, 2010 14:00

    Does it matter what storage array it is ? or it depends on SAS HBA card type in server ?
    Explain please because i have no experience with storage arrays

    As i found here:
    http://www.sun.com/bigadmin/hcl/data/components/details/3196.html
    H200 is supported by Solaris
  • mrokkam1

    23 Posts

    1810

    0

    Posted December 30th, 2010 10:00

    The storage array does determine the required driver. Basically, the RDAC driver is responsible for the following main functions:
    - Direct IO down the correct path(s) when there are multiple paths to a disk
    - Each path to a LUN shows up as a LUN. The driver needs to identify all the multiple paths to a single LUN and combines them to show them as 1 LUN
    - When access to one controller in the array fails, it will send a failover command to the other controller to take ownership of all missing LUNs.

    For this to happen successfully, you need storage array specific drivers. LSI's CTS2600 and the MD3200 arrays are based off the same architecture and I believe there might be a sun version too. So, if you can find the drivers for the equivalent array from any of these vendors, it should work. Then again, these drivers would not be officially supported by Dell. So, it would be up to you to verify the above functionality.

    BTW, the H200 is an internal SAS HBA. How are you connecting to the MD? Or are you connecting it to a 6 Gig SAS HBA from Dell? In either case, both are basic HBAs that provide connectivity between the array and the server.
  • gedd1

    9 Posts

    1810

    0

    Posted December 30th, 2010 12:00

    Thanks for reply !
    I'll try to find drivers you described.
    I think i'm using 6Gbps SAS HBA to connect to the array, but i've to check it to be sure (after backing to work at Monday).

    I've tried to use stmsboot command to enable multipathing and according to documentation:
    (http://docs.sun.com/source/820-3223-14/SASMultipath.html#50511899_78959)
    after reboot i should see new devices (multipathed ? ) but there were no effects.
    Is the reason can be absence of driver You mentioned ?
  • gedd1

    9 Posts

    1810

    0

    Posted January 3rd, 2011 06:00

    I have a solution.
    MD3200 is not supported by default in Solaris 10 u9 so you need to add it as third-party storage device.
    Edit /kernel/drv/scsi_vhci.conf and add following lines:

    device-type-scsi-options-list = "DELL MD32xx", "symmetric-option";
    symmetric-option = 0x1000000;

    Important above is Vendor ID ("DELL" in this case ) have to be 8 chars long, so you need 4 spaces after word "DELL"
    (see also: http://docs.sun.com/app/docs/doc/820-1931/gfjtx?l=en&a=view)

    Then run command stmsboot -e to enable multipathing.
    After reboot you will see new devices (single instance for every LUN, not double)

    One more link, that may be usefull:
    http://docs.sun.com/source/819-0139/ch_3_admin_multi_devices.html

  • dlethe

    16 Posts

    1810

    0

    Posted January 4th, 2011 12:00

    since this is SAS, you would use mpathadm, below shows a SAS disk dual ported in an enclosure

    # mpathadm show lu /dev/rdsk/c1t5000C50020D1DB27d0
    Logical Unit: /dev/rdsk/c1t5000C50020D1DB27d0s2
    mpath-support: libmpscsi_vhci.so
    Vendor: SEAGATE
    Product: ST32000444SS
    Revision: 0005
    Name Type: unknown type
    Name: 5000c50020d1db27
    Asymmetric: no
    Current Load Balance: none
    Logical Unit Group ID: NA
    Auto Failback: on
    Auto Probing: NA

    Paths:
    Initiator Port Name: 500605b001760910
    Target Port Name: 5000c50020d1db25
    Override Path: NA
    Path State: OK
    Disabled: no

    Initiator Port Name: 500605b00175fc90
    Target Port Name: 5000c50020d1db26
    Override Path: NA
    Path State: OK
    Disabled: no

    Target Ports:
    Name: 5000c50020d1db25
    Relative ID: 0

    Name: 5000c50020d1db26
    Relative ID:
    Also for what it is worth, since Dell doesn't have any mechanism to light the LEDs in the enclosures, based on zpool status and query health of the external enclosures, you can check out something at http://www.santools.com/smartmonux

    But -- you need to be using zfs and have the H200 configured for JBOD, not hardware-based RAID. Let zfs do the RAID as it will significantly outperform the PERC and provide the hot snapshots, and all the other goodies that are built into zfs.

  • gedd1

    9 Posts

    1810

    0

    Posted January 5th, 2011 02:00

    Yes, i Know about mpathadm as you can see below:

    >mpathadm show lu /dev/rdsk/c6t6842B2B00070EFC8000003F74D22E0F5d0s2
    Logical Unit: /dev/rdsk/c6t6842B2B00070EFC8000003F74D22E0F5d0s2
    mpath-support: libmpscsi_vhci.so
    Vendor: DELL
    Product: MD32xx
    Revision: 0770
    Name Type: unknown type
    Name: 6842b2b00070efc8000003f74d22e0f5
    Asymmetric: no
    Current Load Balance: round-robin
    Logical Unit Group ID: NA
    Auto Failback: on
    Auto Probing: NA

    Paths:
    Initiator Port Name: 5842b2b0774cb900
    Target Port Name: 5842b2b470efc814
    Override Path: NA
    Path State: OK
    Disabled: no

    Initiator Port Name: 5842b2b0774cb900
    Target Port Name: 5842b2b470ed7f14
    Override Path: NA
    Path State: OK
    Disabled: no

    Target Ports:
    Name: 5842b2b470efc814
    Relative ID: 0

    Name: 5842b2b470ed7f14
    Relative ID: 0

    One thing i noticed - when i start format program every operation takes a looong time, for example disk selecting takes about 1 min. !
    What is tha reason ? I have two LUNs with preferred path via controler 0 of MD3200 and next two via controller 1.
    Maybe here is a problem:
    Current Load Balance: round-robin

    What do you think ?

  • gedd1

    9 Posts

    279

    0

    Posted January 5th, 2011 10:00

    == 3 ==
    MD3200 connected with two cables :
    - port 0 of controller 1 in MD3200 to port 0 of HBA controller in server (1 LUN assigned in storare array to HBAs port 0)
    - port 0 of controller 2 in MD3200 to port 1 of HBA controller in server (1 LUN assigned in storare array to HBAs port 1)

    >mpathadm list initiator-port
    Initiator Port: 5842b2b0774cb900
    Initiator Port: 5842b2b0774cb901

    >mpathadm show lu /dev/rdsk/c3t6842B2B00070EFC80000040D4D232155d0s2
    Logical Unit: /dev/rdsk/c3t6842B2B00070EFC80000040D4D232155d0s2
    mpath-support: libmpscsi_vhci.so
    Vendor: DELL
    Product: MD32xx
    Revision: 0770
    Name Type: unknown type
    Name: 6842b2b00070efc80000040d4d232155
    Asymmetric: no
    Current Load Balance: none
    Logical Unit Group ID: NA
    Auto Failback: off
    Auto Probing: NA

    Paths:
    Initiator Port Name: 5842b2b0774cb900
    Target Port Name: 5842b2b470ed7f14
    Override Path: NA
    Path State: OK
    Disabled: no

    Initiator Port Name: 5842b2b0774cb901
    Target Port Name: 5842b2b470efc814
    Override Path: NA
    Path State: OK
    Disabled: no

    Target Ports:
    Name: 5842b2b470ed7f14
    Relative ID: 0

    Name: 5842b2b470efc814
    Relative ID: 0

    >format -eM
    Searching for disks...done
    AVAILABLE DISK SELECTIONS:
    ...
    4. c3t6842B2B00070ED7F000003F03B853225d0 drive type unknown
    /scsi_vhci/disk@g6842b2b00070ed7f000003f03b853225
    ...
    Disk has unknown type as in situation 2

    Any ideas ?
  • gedd1

    9 Posts

    1810

    0

    Posted January 5th, 2011 10:00

    == 2 ==
    MD3200 connected with two cables :
    - port 0 of controller 1 in MD3200 to port 0 of HBA controller in server (1 LUN assigned in storage array to HBAs port 0)
    - port 0 of controller 2 in MD3200 to port 1 of HBA controller in server (1 LUN NOT assigned in storage array to any HBA's port)

    >mpathadm list initiator-port
    Initiator Port: 5842b2b0774cb900
    Initiator Port: 5842b2b0774cb901

    >mpathadm show lu /dev/rdsk/c3t6842B2B00070EFC80000040D4D232155d0s2
    Logical Unit: /dev/rdsk/c3t6842B2B00070EFC80000040D4D232155d0s2
    mpath-support: libmpscsi_vhci.so
    Vendor: DELL
    Product: MD32xx
    Revision: 0770
    Name Type: unknown type
    Name: 6842b2b00070efc80000040d4d232155
    Asymmetric: no
    Current Load Balance: none
    Logical Unit Group ID: NA
    Auto Failback: off
    Auto Probing: NA

    Paths:
    Initiator Port Name: 5842b2b0774cb900
    Target Port Name: 5842b2b470ed7f14
    Override Path: NA
    Path State: OK
    Disabled: no

    Target Ports:
    Name: 5842b2b470ed7f14
    Relative ID: 0

    >format -eM
    Searching for disks...done
    AVAILABLE DISK SELECTIONS:
    ...
    4. c3t6842B2B00070ED7F000003F03B853225d0 drive type unknown
    /scsi_vhci/disk@g6842b2b00070ed7f000003f03b853225
    ...

    Disk has unknown type and i can do nothing with it (format/label, fdisk fails) for example:
    >fdisk /dev/rdsk/c3t6842B2B00070EFC80000040D4D232155d0p0
    Error in ioctl DKIOCGMBOOT: I/O error
    fdisk: Error reading partition table from /dev/rdsk/c3t6842B2B00070EFC80000040D4D232155d0p0.
  • gedd1

    9 Posts

    1810

    0

    Posted January 5th, 2011 10:00

    One more strange problem:
    Look at 3 situations - i'll describe them in 3 separate posts because of characters limit.
    Multipathing in Solaris 10 is enabled as described in previous posts.

    == 1 ==
    MD3200 connected with one cable :
    - port 0 of controller 1 in MD3200 to port 0 of HBA controller in server (1 LUN assigned in storage array to HBA's port 0)

    >mpathadm list initiator-port
    Initiator Port: 5842b2b0774cb900

    >mpathadm show lu /dev/rdsk/c3t6842B2B00070EFC80000040D4D232155d0s2
    Logical Unit: /dev/rdsk/c3t6842B2B00070EFC80000040D4D232155d0s2
    mpath-support: libmpscsi_vhci.so
    Vendor: DELL
    Product: MD32xx
    Revision: 0770
    Name Type: unknown type
    Name: 6842b2b00070efc80000040d4d232155
    Asymmetric: no
    Current Load Balance: none
    Logical Unit Group ID: NA
    Auto Failback: off
    Auto Probing: NA

    Paths:
    Initiator Port Name: 5842b2b0774cb900
    Target Port Name: 5842b2b470ed7f14
    Override Path: NA
    Path State: OK
    Disabled: no

    Target Ports:
    Name: 5842b2b470ed7f14
    Relative ID: 0

    >format -eM
    Searching for disks...done
    AVAILABLE DISK SELECTIONS:
    ...
    4. c3t6842B2B00070ED7F000003F03B853225d0 DEFAULT cyl 36404 alt 2 hd 255 sec 63
    /scsi_vhci/disk@g6842b2b00070ed7f000003f03b853225
    ...

    Everything is OK