Larger LUN size considerations

Any datastore you want to replicate via RecoverPoint has to be (2TB - 512 bytes) or smaller.  Like you we're using larger LUNs / datastores but for replicated datastores just under 2TB.

Note: That info is as far as RP 3.4 - 3.5 is the latest and may be different.

J

Interesting point.

Best practices seem to point to increasing LUN sizes. I get that.  Bigger LUNs = Less LUNs = Less administration = better for admins. However, there used to be quite some discussion about queue depths when picking a LUN size, as there is a queue per LUN. What i want to know is if we somehow need to think different of queues these days ? Less and bigger LUNs means more VMs will share the same queues, given VMs dont get much bigger most of the time. Is this less of an issue these days with the current versions of VMware?

Hi,

if you wouldn't mind let me add my 2 cents here.

Keep in mind what virtualisation do and don't do, any one of the things it doesn't do is virtualizing IO's.

What I mean is the following, keep in mind that your infrastructure has to handle the total IO load, regardless if you're using VM's or not.

In good old days each server had multiple disk, (and sometimes for good reasons ), now many VMware customers keep this design but place all vmdk's on one datastore which is contra productive.

VAAI itself is cool, but not often used when you compare it to normal IO's.

SDRS (and storage array technologies like FAST) tries to get rid of the problems with IO performance issues, but they do require analysis time and are therefor reactive tools.

In general, a good design is priceless.

So you should keep an eye on the amount of IO's your environment produces and then present enough LUN's to handle that load, all other features will than take care that the load will be moved when needed.

Hth,

Ralf

Clustor,

Just remember you're able to do multiple LUNs presented as 1 datastore - so if you find that your LUN queue is a bottleneck, there's little downside to creating 4 1 TB LUNs and presenting them as 1 4TB datastore.

Ralf's point is in general good advice, but really has nothing to do with our conversation. Assuming you're talking about whether it makes sense to put the 4x1TB or 1x4TB LUNs all on the same RAID group (going with traditional LUNs to make the conversation easy), they both provide the same amount of IOPs. As mentioned you do get a queue by LUN, so there is that.

VMware performance can frequently be a bit of a whack-a-mole situation. A good design is very important, no argument, but understanding how to identify and resolve bottlenecks when they occur is vitally important.. So monitor the queues for your Datastores / LUNs and as necessary re-design your layout to reduce the hot spots.

I'd suggest checking out the joint EMC / VMware session from VMworld this year on Storage best practices.

http://virtualstorageguy.com/2012/09/10/vmworld-2012-sto2980-vsphere-storage-best-practices/

Although no one mentioned heap size anywhere here, you do bring up a good point. Heap size, as I understand it, only matters when you're talking about the aggregate amount of storage that makes up the VMs that are attached to the host (aka, if you have a 100G VM on a 4TB datastore attached to an ESX host, the heap on the ESX host only cares about the 100G)

Having said that, although the defaults and maximums  for vSphere 4 and vSphere 5 changed, there is a workaround

http://www.boche.net/blog/index.php/2012/09/12/monster-vms-esxi-heap-size-trouble-in-storage-paradise/

Also check out KB article

KB article 1004424: An ESXi/ESX host reports VMFS heap warnings when hosting virtual machines that collectively use 4 TB or 20 TB of virtual disk storage.

The key is to change VMFS3.MaxHeapSizeMB

Note also this only is an issue with VMFS datastores, not NFS.

BTW, i don't think your numbers are correct on the blocksizes for v4 and v5.

VMFS5 uses 1MB default. Earlier versions VMFS used 1,2,4 or 8MB

Clustor,

I think you're right about the block size. The main point I wanted to make is that maximum addressable open storage went down between ESX v4 to v5.  So be careful.

◦In ESXi/ESX 4.x and earlier releases, max heap size is 128 MB. This allows a maximum of 32 TB of open storage.

◦In ESXi 5.x, max heap size is 256 MB. This allows a maximum of 25 TB of open storage.

Jay,

Thanks for the additional info. If people are using 4 TB LUNS, they may have already followed the workaround to bump up their ESX hosts to the maximum heap size.  I know we have. We'd love to go way beyond what ESX allows. It's kind of a thorn right now and the kb article and workaround link attest to that. You have to be really careful when you put one of your ESX servers in maintenance mode. You could easily go beyond the max heap size on the other ESX servers in the cluster when everything gets vmotioned off the ESX server that you put in maint mode.

Jay just to correct a statement you made earlier. The RP limit is 32TB for a lun not 2TB assuming you are using one of the array based spitters.