Deduplication on block LUNs for VMware

Please see KB 188409 - this is a list of KB Articles about Deduplication and best practices for setting up deduplication.

You should also read the White Paper for Depuplication prior to setting up dedup. There are restrictions that you should be aware for first. Also, see this document for a quick overview of deduplication:

glen

It's not necessarily best practice, but an optional practice

There seems to be a fair amount of angst on the implementation of dedup in VNX due to there being numerous identified issues experienced by early adopters. Each new release of block OE seems to bring a raft of fixes and updates.

Keep in mind the following points;

• All dedup'd LUN's are located into a new deduplication 'Container' owned by a single SP. This means all dedup'd LUNS will be owned by the same SP, which can require management/intervention to avoid saturating this single SP
• It is post process, not inline and uses thin LUN's. Any thick LUN's you enable dedup on will be migrated to thin.
• There may be a performance impact, there is definitely an overhead.
• Update to the latest Block OE to ensure you have the most up-to-date featureset
• test on some non-tier1/non-mission critical LUN's

Check out the guide for some good reference  https://www.emc.com/collateral/white-papers/h12209-vnx-deduplication-compression-wp.pdf

In theory, vm's are a good candidate for space savings, but like always, test thoroughly.

EDIT: all these empty auto-responses from 'beautifial' are getting a bit nuts!

Thanks Glen

Brett@S wrote: It's not necessarily best practice, but an optional practice There seems to be a fair amount of angst on the implementation of dedup in VNX due to there being numerous identified issues experienced by early adopters. Each new release of block OE seems to bring a raft of fixes and updates. Keep in mind the following points; All dedup'd LUN's are located into a new deduplication 'Container' owned by a single SP. This means all dedup'd LUNS will be owned by the same SP, which can require management/intervention to avoid saturating this single SP

to add on Brett's last point, if you have multiple pools than you will be able to utilize multiple SPs. One pool's lun will be stacked on SPA and other on SPB. I still hate this  because it will be come so imbalanced.  No dedupe on VNX for me anytime soon, this should not have been released in its current state. Who in their right mind would want to place all their LUNs on one SP.

Thanks Brett

I am not going with deduplication for block on VNX then, at least not for production.

+1 to Brett and Dynamox.

We had multiple issues with Deduped LUNs from our VNX2 series arrays with LUNs presented to ESX hosts - and once we migrated them to different pools (and re-balanced all pools out), the issues no longer persist.

It is definitely a deal breaker with assigning LUNs per SP for utilizing the dedupe functionalities in the VNX arrays. Until a fix is available, I wouldn't want to venture into using it.

My $0.02

Hi akc5247,

Can you describe the issues you experienced with dedupe luns presented to ESX hosts?  We are looking to test this setup and any info you could pass along would be greatly helpful.

Re,

-F

Be sure to read all the documents that have been attached to this discussion. One major point is the amount of Writes to the LUNs - if you get over 30%, performance will suffer. You really need to know what the IO profile of the LUNs will be before you try deduping them.

glen

Hi fvo,

We started our VNX5400 usage by enabling Dedup on most of the LUNs (VMware).

But dedup causes quite big performance issues, both write and read. Especially the allocation of new data slices within dedup container is quite problematic currently, it doesn't spread out to whole pool/tier in same way like Thick LUNs.

Due to these performance issues, we had to move away from dedup/thin LUNs to using Thick LUNs. Only using DeDup for some EOL/Archive date.

Also, there is still open defect where dedup process can make autotier rebalancing fail.

-sk

glen & sk,

Thank you for the feedback.  It is extremely appreciated.  We have a 5600 and wanted to test the tech against our vmware env.  Knowing this we will proceed with extreme caution. I have already spoken with our team and we are already leaning towards going with out the tech for production.  We do however use thin luns off the vnx for our vmware env.  We have been doing so since vsphere 4.1 without issue (*knock on wood).

Thanks again guys!

Agreed, dedup is still not quite right. No issues with thin LUN's, I use them extensively also and they're solid. (As long as you watch the %Consumed !)

When you say used for VMware are we talking all of the VM data? Has any considered/tested only putting the OS disks (c: drive) within the dedupe container? Wonder if that would meet the performance but also provide some space savings?