VNX thin pool with VMware datastore, space reclaimation

L_L,

Firstly, let me just quickly ask (apologies if I'm stating the obvious). 

1) When you mention that you recreated the datastores on the array, can I verify also that you deleted the underlying thin LUN and recreated them new once you vacated them?

From the underlying pool perspective, that will free up the allocated slices.  I'm sure this is the case, but wanted to make sure you weren't simply deleting the datastore and reusing the LUN to recreate your "smaller" datastores.  Normally (and is best practice), you have a one-for-one relationship between LUN and datastore; however, I have seen people put more than one datastore on a LUN so I want to make sure.

2) As you correctly noted, since you are only running ESX/ESXi 4.1, the reclaim feature (also requires compatible VNX OE for Block) via vmkfstools is not available to you.  However, I simply wanted to make reference to a post once you do upgrade:

https://community.emc.com/thread/162928

3) The proper way to reclaim space with your ESX/ESXi 4.1 environment would be as follows:

NOTE: I am assuming that you are provisioning as THICK on THIN.  Also, keep in mind assuming you have the capacity, this can also be performed intra-array, and if configured properly you would get the benefit of the VAAI XCOPY primitive during the svMotion; however, I assume you are doing it to/from andother array since you are limited in space.  Just something to think about.

a) Since there are multiple layers here and you have to make each layer aware of the unused blocks above it, you need to first reclaim the space within the guest image. 

For instance, if it is a Windows environment even though you delete a file, those NTFS clusters are not zeroed out (which is also as we all know why you can "undelete" a file).  Therefore, your first strategy should be to zero out the free space in the guest image.  This is also important for the next step during the svMotion.  To do so you could run a utility such as the free sdelete originally from SysInternals:

sdelete -c

usage: sdelete.exe [-p passes] [-s] [-q]

       sdelete.exe [-p passes] [-z|-c] [drive letter]

   -c         Zero free space (good for virtual disk optimization)

   -p passes  Specifies number of overwrite passes (default is 1)

   -q         Don't print errors (Quiet)

   -s         Recurse subdirectories

   -z         Clean free space

WARNING: This will fully inflate the underlying vDisk and in turn the corresponding slices of the LUN as it is writes to each block

b) Then, when performing the svMotion, when prompted, you would select "Thin Provisioned Format". This will reclaim the space at the VMFS layer

c) Next, delete the original datastore and the underlying LUN.  You will see the allocated capacity of the pool drop accordingly

d) Recreate the thin LUNs and the datastores (bp is to do one-for-one allocation).  For each thin LUN you recreate, you should see the allocated capacity of the pool increase by only 3GB (initial overhead).

e) Then svMotion back to the new datastores but this time select "Thick Provisioned Format" for the type

NOTE: In ESX/ESXi 4.1, from the GUI, this will perform a (Lazy)ZeroedThick meaning blocks are zeroed only when they are written to which is important for the underlying thin lun.  With ESXi 5, you will see Thick Provisioned Format separated into: Lazy Zeroed and Eager Zeroed in the migration wizard.  Just remember, selecting EagerZeroedThick is just as if you did a full format in Windows which isn't thin friendly (inflates that portion of the thin lun), but in a VMware environment it is required for FT enabled pairs.  It can also be a strategy to get the best performance initially as there isn't the overhead of zeroing the blocks as they are written to.

I hope this helps.