Announcement Banner
UNSOLVED

hod2011

updated

14 years ago

H

hod2011

6 Posts

0

26546

January 14th, 2013 04:00

MD3000i slow performance

Hi

I have taken over this environment from previous admin.

I am new to SAN's and was looking for advice on where to begin to troubleshoot a performance issue we are having with our MD3000i

We have a MD30001 with 15 disks(8 SAS, 7 SATA)

We have dual raid controller and all 4 ISCSI ports are configured and are connecting to two power connect 2716 switches.

Our VMware host machines have 4 nics, two for connecting to the network and two ISCSI data which are connected to the ISCSI switch.

Lately our backups which backup to a BTD LUN on the MD3000i are taking twice as long as usual.

We are also seeing an issue where we can no VMotion vms as they time out after a couple of hours.

It was metioned to me that the ISCSI switchs may need to be restarted, i have done this but it has had no impact.

I did a quick check and Jumbo Frams is disabled on the switches, and MD3000i

I was wondering where is the best place for a newb to start troubleshooting?

Thanks

Hod2011

  • Dev Mgr

    6 Operator

    •

    9308 Posts

    •

    3 Points

    478

    0

    Posted January 14th, 2013 08:00

    There are several possible issues going on here.

    A few pointers:

    - The PowerConnect 2700 series isn't really a decent switch for iSCSI traffic, so they may be part of your issues

    - Don't upgrade to vSphere 5 as this hasn't been validated with the MD3000-series (only validated with the newer MD3200 and 3600 series)

    - which vmkernel are you using for vMotion? Is it one of the iSCSI vmkernels or another vmkernel that is leveraging the iSCSI vSwitch(es) or the iSCSI physical switches?

    If possible, can you SSH into the server and post the output of:

    - esxcfg-vswitch -l

    - esxcfg-vmknic -l

    and then provide the 4 IP addresses that are being used for the iSCSI ports on your MD3000i?

    E.g.:

    Controller 0 iSCSI port 0: 192.168.130.101

    Controller 0 iSCSI port 1: 192.168.131.101

    Controller 1 iSCSI port 0: 192.168.130.102

    Controller 1 iSCSI port 1: 192.168.131.102

  • hod2011

    6 Posts

    478

    0

    Posted January 15th, 2013 03:00

    Dev Mgr, thanks for the advice on vSphere 5.

    I not sure if there would be funds to replace the power connect switches, but if we were to look at doing this what would you recommend?

    Vmotion is handled by the management network which is on a different subnet and which is connected to seperate network switch.

    Here are the results of the two commands you asked me to run.

    ~ # esxcfg-vswitch -l

    Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks

    vSwitch0         64          8           64                1500    vmnic3,vmnic2

     PortGroup Name        VLAN ID  Used Ports  Uplinks

     VM Network            0        4           vmnic3,vmnic2

     Management Network    0        1           vmnic3,vmnic2

    Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks

    vSwitch1         64          3           64                1500    vmnic0

     PortGroup Name        VLAN ID  Used Ports  Uplinks

     iSCSI_0               0        1           vmnic0

    Switch Name      Num Ports   Used Ports  Configured Ports  MTU     Uplinks

    vSwitch2         64          3           64                1500    vmnic1

     PortGroup Name        VLAN ID  Used Ports  Uplinks

     iSCSI_1               0        1           vmnic1

    ~ #

    ~ # esxcfg-vmknic -l

    Interface  Port Group/DVPort   IP Family IP Address                              Netmask         Broadcast       MAC Address       MTU     TSO MSS   Enabled Type

    vmk0       Management Network  IPv4      192.168.196.xx                          255.255.252.0   192.168.199.255 00:1d:09:73:b2:be 1500    65535     true    STATIC

    vmk1       iSCSI_0             IPv4      10.10.0.x                             255.255.255.0   10.10.0.255     xx:xx:xx:xx:xx:xx 1500    65535     true    STATIC

    vmk2       iSCSI_1             IPv4      10.10.1.x                              255.255.255.0   10.10.1.255     xx:xx:xx:xx:xx:xx 1500    65535     true    STATIC

    Here are the four IP addresses which are being used on the MD3000i

    Controller 0 iSCSI port 0: 10.10.0.1

    Controller 0 iSCSI port 1: 10.10.1.1

    Controller 1 iSCSI port 0: 10.10.0.2

    Controller 1 iSCSI port 1: 10.10.1.2

  • Dev Mgr

    6 Operator

    •

    9308 Posts

    •

    3 Points

    478

    0

    Posted January 17th, 2013 09:00

    Replacing SATA with SAS drives does mean getting 10k or 15k SAS drives. Dell also offers 'near-line SAS', which is still 7200rpm and wouldn't make a difference in performance from SATA drives.

  • hod2011

    6 Posts

    478

    0

    Posted January 17th, 2013 09:00

    I also logged a call with DELL and they took a look at the MD3000i logs. They said that their was some I/O queuing on the LUNS which were running SATA drives. They advised that the SATA drives cannot handle the I/O and that we should look to replace them with SAS or try to move our more data intensive VM's onto the SAS LUNS.

  • hod2011

    6 Posts

    478

    0

    Posted January 18th, 2013 05:00

    Apart from replacing the drives with faster SAS drives, is there anything else that i should check that could be impacting the performance?