jingyi1
3 Argentium

How will ESX handle Virtual Provisioned LUN

Jump to solution

I created several thin LUNs from VMAX for VMware ESX host because I want to test FAST VP. After adding these LUNs to ESX storage group, some questions arised on how VMware handle thin LUNs.

1. If VASA is not configured, will ESX be aware of this is a thin LUN not? My guess is no.

2. If ESX doesn't know it's thin LUN, will VMFS datastore creation allocate all space of the thin device? My guess is yes

3. Is there a way not to allocate the whole device capacity? Currently I  create a fairly small, 6G VMFS datastore from thin device which can be expanded later. Will only the 6G be allocated? What if we use RDM, does RDM work?

4. If VASA is configured, what different does it make for thin device provision, Datastore creation?

Your inputs are welcomed.

0 Kudos
1 Solution

Accepted Solutions
codyhosterman
3 Argentium

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution

ESXi is in fact aware if a LUN is thin without VASA, ESX will identify thin provisioned LUNs using thin provisioning enabled (TPE) bit from READ CAPACITY (16) response. This is of course the arrays are T10 compliant which EMC arrays are.

It is correct that VASA is mainly for administrative purposes, but there are a few other things to note. Namely the fact that there is an extra alert activated by the VASA Provider that passes along an alert when the thin device hits a certain percent full on the array (the thin device itself, not the pool). If this alert is sent along to vCenter any new provisioning on that VMFS is prevented until the situation is cleared (clearing the thin LUN somewhat, increasing the size of the thin lun etc...)

My white paper that dynamox pointed out should answer your other questions.

16 Replies
LouisLu
3 Argentium

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution

Good questions. I am interested in these questions also. Eager for answers.

0 Kudos
dynamox
6 Thallium

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution
RafaelNovo
2 Iron

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution

1. Yes, ESX will not be aware of thinlun

2. No, creating a VMFS volume will not allocate the entire LUN. ESX will only allocate disk space when you actually USE the space. Exception for the Eagerzeroed thick VMDK that will allocate all disk space upfront.

3. Answer above.

4. None. VASA is used for administration benefit. (basically VM placement/migration decision). BUT you want to have VAAI. VAAI will enable ThinProvisioning stun that will put VM "on hold" when storage runs out of disks space.

0 Kudos
codyhosterman
3 Argentium

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution

ESXi is in fact aware if a LUN is thin without VASA, ESX will identify thin provisioned LUNs using thin provisioning enabled (TPE) bit from READ CAPACITY (16) response. This is of course the arrays are T10 compliant which EMC arrays are.

It is correct that VASA is mainly for administrative purposes, but there are a few other things to note. Namely the fact that there is an extra alert activated by the VASA Provider that passes along an alert when the thin device hits a certain percent full on the array (the thin device itself, not the pool). If this alert is sent along to vCenter any new provisioning on that VMFS is prevented until the situation is cleared (clearing the thin LUN somewhat, increasing the size of the thin lun etc...)

My white paper that dynamox pointed out should answer your other questions.

jingyi1
3 Argentium

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution

Thanks all for your response.

Cody, I'm reading your WP and will check the VMAX pool  to verify allocation for different provisions ESXi use. Will update everyone of my findings.

0 Kudos
jingyi1
3 Argentium

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution

I have read Cody's WP and tested it on the lab box. Amazing document, it has answered some questions that puzzled me for a while. As below are answers for all questions.

1. If VASA is not configured, will ESX be aware of this is a thin LUN not? My guess is no.

Yes, ESX will identify thin provisioned LUNs using thin provisioning enabled (TPE) bit from READ CAPACITY (16) response. This is of course the arrays are T10 compliant which EMC arrays are.

2. If ESX doesn't know it's thin LUN, will VMFS datastore creation allocate all space of the thin device? My guess is yes

ESX knows it's thin LUN. RDM, Thick Provision Lazy Zeroed, and Thin Provision will only allocate the meta data from pool. Only the creation of Thick Provision Eager Zeroed Datastore will allocate all assigned space from thin pool to ESX. The Metadata consumptionis 970MB to be reserved and 1MB for each 100GB in VMFS5 volumes. Even Metadata does not be written compltely at the volume creation.

3. Is there a way not to allocate the whole device capacity? Currently I create a fairly small, 6G VMFS datastore from thin device which can be expanded later. Will only the 6G be allocated? What if we use RDM, does RDM work?

Same as Q2.

4. If VASA is configured, what different does it make for thin device provision, Datastore creation?

VASA can pass Symmetrix thin pool  out-of-space warning to Vcenter so that ESXi can handle storage accordingly.

It's recommended to create virtual VMFS datastore on thick Symmetrix Lun and create Thick Provision Lazy Zeroed datastore on thin device. thick on thin  or thin on thick model. Zeroedthick lun creation will allocate space on ESX level, but won't initialize with zero on storage level, therefore no track need to be allocated on thin pool.

I also find after adding a thin VMDK drive to VM,   Windows XP format utility (with or without quick option)  won't allocate any space on storage thin pool at all. Initiailizing and partitioning this drive will write small amount of metadata to disk.

Creating a Thick Provision Eager Zeroed Datastore will allocate all assigned space in thin pool. However it can be freed up after Datastore creation by SMC. I do be confused by  "free" and "reclaim" options in SMC "Start Allocate/Free/Reclaim". Reclaim only unallocate very few tracks and Free seems unallocate the whole disk zero written tracks. What do these 2 really mean?

0 Kudos
Highlighted
vforde
2 Iron

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution

Hi,

Jingyi, I too am slightly confused

I am not sure what version of Solutions Enabler/SMC you are on but it seems to have slightly changed in 7.3. In 7.3 it is not documented as well as previous versions IMO.

In previous versions there was a reclaim_type=unwritten (allocated but not used) or reclaim_type=reclaim (which reclaim unwritten and devs zeroed out). There is only a "start free" statement there is no start reclaim

In 7.3 the support for the clause reclaim_type=<unwritten|reclaim> seems to be lapsed.

My reading is that instead there is "start free" or "start reclaim" where "start free" reclaims the equivalent to unwritten (allocated but not used) and also does not have persistent attribute set. "start reclaim" reclaims unwritten even if the persistent attribute set. I am assuming that "start reclaim" reclaims zeroed out space as well as unwritten even though I can not see this explicitly in the documentation.

The best practices documentation (P/N 300-006-718) references the symconfigure syntax with reclaim_type=<unwritten|reclaim> so could it be clarified for SE/SMC 7.3? 

Regards,

Victor

0 Kudos
codyhosterman
3 Argentium

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution

Starting with Solutions Enabler V7.3, two new commands, start reclaim and stopreclaim are introduced to manage reclaiming thin device tracks. The start free and stop free commands will continue to be supported for unwritten allocations on a range of cylinders for a device, and for stopping the free operation on a thindevice in the De-allocating state (only).

To reclaim tracks on thin devices, use the following syntax:

start reclaim on tdev <SymDevName[:SymDevName] |in DG DgName | in SG SgName>[allocate_type = persistent];

If the allocate_type=persistent is specified, this command frees up tracks that

are unwritten or zero-based, even if they are marked as persistent.

So start free only de-allocates tracks that are allocated but also set Not Written By Host (NBWH). Start reclaim will reclaim both allocated tracks with NWBH and zeroed extents.

vforde
2 Iron

Re: How will ESX handle Virtual Provisioned LUN

Jump to solution

Thanks for clarifying Cody, it is crystal clear now the way 7.3 handles it!

Regards,

Victor

0 Kudos