fa-jakes
1 Copper

MD3460 and Dell 12Gbps SAS HBA

Recently purchased a Dell MD3460 and a couple of R730 servers each with a pair of Dell 12Gbps SAS HBA cards in them.  This is going to be a new virtual environment using ESXi v6.  We have a very similar setup using a Dell MD3200 and the Dell 6GBps SAS cards and everything in that environment setup out of the box perfectly.  Not so with the new MD3460 array and the Dell 12Gbps SAS cards.  By default ESXi loads up the LSI-MSGPT3 driver for the Dell 12Gbps SAS HBAs which is wrong.  You need to switch over to the MPT3SAS driver.  Once you start using the MPT3SAS drivers if you're using multipathing for redundancy to the MD3460 all the paths on one of the cards will show up as Dead in vSphere.  That's not correct.  To fix that issue you need to set the SATP plugin for the MD3460 to VMW_SATP_LSI and set the default path selection policy to Round Robin.  None of this is noted in the manual(s) and the Vmware Compatibility Guide it says to use the VMW_SATP_ALUA plugin which is wrong.

I beat my head against this problem for a couple of days and after engaging several Dell and Vmware techs to no avail I figured I'd share this so hopefully others will find it helpful.  Here are exact steps I followed to fix the problems noted above...

Dell SAS 12Gbps HBA External cards with an MD3460 SAN

  • *Download MPT3SAS v11 drivers from https://my.vmware.com/group/vmware/details?downloadGroup=DT-ESXI60-LSI-MPT3SAS-11000000-1OEM&product...
  • Unzip and copy mpt3sas-11.00.00.00-6.0-offline_bundle-3035604.zip up to local store on the Vmware ESXi host.
  • Enable SSH on the ESXi host and open an SSH session.
  • Run the following command on each ESXi host to uninstall the default LSI driver: esxcli software vib remove –n lsi-msgpt3
  • Run the following command to install the new mpt3sas driver: esxcli software vib install –d /vmfs/volumes/datastore1/mpt3sas-11.00.00.00-6.0-offline_bundle-3035604.zip
  • Run the following command to change the SATP plugin and PSP selection to VMW_SATP_LSI and Round Robin: esxcli storage nmp satp rule add -V DELL -M "MD34xx" -P VMW_PSP_RR -s VMW_SATP_LSI
  • Reboot the ESXi host(s)
  • Open the PowerVault Modular Disk Storage Manager.  Click on the “Stroage & Copy Services” tab.  Right click on one of your disks and go to Advanced -> Redistribute Virtual Disks.  If this option isn’t available then all the virtual disks are already on their preferred paths (which is good).
  • Connect to the ESXi host via the vSphere Client.
  • Go to the Configuration tab and in Hardware select the Storage Adapters.
  • Click on “Rescall All” at the top.
  • Find the cards listed under “Avago (LSI) 3008”.  Click on the first one and then click on “Paths”.  You should see some targets lists as “Active (I/O)” and some listed as “Stand by”.  When you click on the other adapter the targets that were in Stand by on the first card should be set to “Active (I/O)” on the second card.

*I don’t think that the driver has to be v11.  I think that v8 or any other version of the mpt3sas driver would probably work as well, I just landed on v11 and being the most recent version is the one I’ll stick with.  The real secret is removing the default LSI_MSGPT3 driver and using an mpt3sas driver with “VMW_SATP_LSI” and “Round Robin”.