Management Networks and SAN Snapshotting

Thank you so much for your input, Don. I really appreciate it. Your answer regarding multiple volumes makes complete sense to me. Since I'm using multipathing (via VMware round robin) I'm assuming I need to balance my number of iSCSI connections to the SAN with the performance benefits offered by multiple volumes. I suspect this won't be as much of an issue with current firmware versus older (I know the iSCSI connection number limit was doubled in recent firmware).

I already have SanHQ 2.20 installed so I can use that to look at my current queue depths. I also did some poking around the forums and I see there is some LUN design talk as it relates to VMware. I understand there is no one-size-fits-all solution so I'll tackle all that next.

If I see in SANHQ 2.20 that my queue level never hits 32 do you imagine I would see less performance benefits to chopping my LUNs up into smaller pieces? I imagine more iSCSI connections at the same time is better, I just want to make sure that the pain of redesigning and copying all my VMs into additional smaller LUNs is going to be worth it.

Should I limit my redesign scope to focus on servers that use the most IOPS? It'd basically be my single Exchange 2010 box, two separate SQL servers, and my file server.

Thanks for all the great info, excellent post. I'm using vSphere 5 Standard. I did some testing with the iops value when I first got my SANs and setting it to 3 didn't improve my results with iometer (using settings to take advantage of the 1MB block size) so I didn't keep the change. I understand now why I didn't see improvement with iometer - I knew the results wouldn't be 'real world' but I didn't know how else to test. I'm glad the consensus is to set it to 3.

Now I've set my iops value to 3, disabled LRO, and turned delayed ACK off on all my hosts. The LRO thing was new to me so thanks for mentioning it! I know there is a lot of information out there about lun design and sql/exchange best practices so I'll read up on that stuff next.

Thanks again for taking the time to reply.

You are very, very welcome.   I'm glad I could help.

Those would actually create a support case via the Equallogic Website. Alerts from SANHQ do not.

FYI:  Here's a link to the port redirector, rinetd, that I mentioned.

codewut.de/Port-Redirection-with-Windows

Hey Don, can you confirm that this is still advisable in vSphere 5.0 when using Round Robin?  (I dug around some Dell / EQL docs, but it didn't mention it.  For some reason I thought this was no longer an issue.  You covered the LRO command line change, but what was the string needed for setting the IOPs to 3 (under vsphere 5)?  What have you found as the best way to monitor the balance of the RR?

Yes, ESXi v5 uses the same 1000 IOs setting by default, so it needs to be changed.  Especially when you have more than two interfaces for iSCSI and multiple member pools.

Setting default policy for EQL devices to Round Robin. So new volumes discovered will be set to Round Robin

#esxcli storage nmp satp set --default-psp=VMW_PSP_RR --satp=VMW_SATP_EQL

**These new volumes will still need to have the IOPs value changed.

To gather a list of devices use:

#esxcli storage nmp device list

You'll need the naa. that corresponds to the EQL volumes in that list. That the "device number" that is used in the next command.

Existing volumes can be changed to Round Robin

#esxcli storage nmp device set -d naa.6090a098703e30ced7dcc413d201303e --psp=VMW_PSP_RR

You can set how many IOs are sent down one path before switching to the next. This is akin to rr_min_io under Linux.

NOTE: This will only work if the policy has been changed to Round Robin ahead of time.

The "naa.XXXXXXXXXXXXX" is the MPIO device name.

You can get a list of devices with:

#esxcli storage nmp device list

naa.6090a098703e5059e3e2e483c401f002

Device Display Name: EQLOGIC iSCSI Disk (naa.6090a098703e5059e3e2e483c401f002)

Storage Array Type: VMW_SATP_EQL

Storage Array Type Device Config: SATP VMW_SATP_EQL does not support device configuration.

Path Selection Policy: VMW_PSP_RR

Path Selection Policy Device Config: {policy=iops,iops=3,bytes=10485760,useANO=0;lastPathIndex=3: NumIOsPending=0,numBytesPending=0}

Path Selection Policy Device Custom Config:

Working Paths: vmhba36:C0:T1:L0, vmhba36:C1:T1:L0, vmhba36:C2:T1:L0, vmhba36:C3:T1:L0

This also lets you confirm the path policy "VMW_PSP_RR" Which is VMware, Path Selection Policy, Round Robin" And not the IOPs value has already been set to '3'.

#esxcli storage nmp psp roundrobin deviceconfig set -d naa.6090a098703e30ced7dcc413d201303e -I 3 -t iops

#esxcli storage nmp psp roundrobin deviceconfig get -d naa.6090a098703e30ced7dcc413d201303e

Byte Limit: 10485760

Device: naa.6090a098703e30ced7dcc413d201303e

IOOperation Limit: 3

Limit Type: Iops

Use Active Unoptimized Paths: false

For monitoring I use a combination of the performance monitoring in ESXi and SANHQ.  

Wow... I overlooked this somehow.  Thanks Don.  Having previously set each datastore to RR, I did execute the following on one of my datastores:

#esxcli storage nmp psp roundrobin deviceconfig set -d naa.6090a09860213c7843eb34e9380180de -I 3 -t iops

Looking at the details again via esxcli storage nmp device list, I see that the "iops=3" validated the change, but I also see the the string "policy=rr" was changed to "policy=iops"  Is this expected?

Also, is it advisable that this only be performed while in maintenance mode, or can it be changed while in production?

Perfect.  The test I had done was on a system in maintenance mode.  Just wanted to verify.  Thanks.

Yes policy=iops is correct.  From the solution I included:  

     Path Selection Policy Device Config: {policy=iops,iops=3,bytes=10485760,useANO=0;lastPathIndex=3:

It 'can' be done live but I prefer not to personally.   Just in case the value gets set to something wrong and performance is impacted.  

You're welcome!

Thanks for the link to rinetd, Don. I plan on forwarding NTP and SMTP traffic from my isolated iscsi network onto my production network via my physical backup server. Seems like a better idea than routing that traffic...why make things more complicated than they need to be, right?

that works, or since you have a 4000 there's a dedicated Mgmt port @ 100Mb that you can put on your LAN so you can manage it and get alerts to your Exchange server.