Thin provisioning is a tool that allows an administrator to define quotas that exceed the capacity of the cluster. Doing this accomplishes two things:
1. It allows a smaller initial purchase of capacity, and the ability to simply add more as needed, promoting a capacity on demand model.
2. It allows the administrator to set larger quotas initially and not have to continually increase them as users consume their allocated capacity.
With a quota that exceeds the cluster capacity, there is nothing to stop users from consuming all available space and causing service outages fro all the users and services using the cluster.
Deduplication is applied at the subdirectory level and targets all files and directories underneath one or more root directories. As you write file to the cluster, some of those files or blocks of data in the files might be duplicates. You can run a deduplication job that scans the file system to see if the data already exits. After duplicate blocks are discovered, SmartDedupe moves a single copy of those blocks to a special set of files known as shadow stores. During this process, duplicate blocks are removed from the actual files and replaced with pointers to the shadow stores.
With post-process deduplication, new data is first stored on the storage device and then a subsequent process analyzes the data looking for commonality. This means that initial file write or modify performance is not impacted because no additional computation is required in the write path.
The deduplication job is set to run at low priority by default, so impact to your workflow should be minimal. However, it’s a good idea to wait until users have finished modifying their files on the cluster before you run the job.
Finally, please consider moving this question as-is (no need to recreate) to the proper forum for maximum visibility. Questions written to the users' own "Discussions" space don't get the same amount of attention and questions can go unanswered for a long time.
You can do so by selecting "Move" under ACTIONS along the upper-right. Then search for and select: "Isilon" which would be the most relevant for this question.
Does SmartQuota on Isilon calculate quota based on file size or real data usage? For example, with a thin-provisioned VMware virtual disk file of 10GB capacity and 5GB usage, will the SmartQuota report 10GB or 5GB?
A1: Smartquota will take that as 5gb + overhead if included while setting up quota on the path.
2. Deduplication
Looks like this feature is added in OneFS v7.1 with SmartDedupe. Are there any impacts on VMware virtual machines from running SmartDedupe? How much space saving (percent) SmartDedupe usualy gains?
A2: Dedupe calculation is tough though dedupe scans a directory for redundant data blocks and deduplicates all redundant data stored in the directory. Available only if you activate a SmartDedupe license.
Hence there is no fixed calculation for dedupe. For more detail refer to document link attached and look for Smartquota & deduplication.
Jeffey1
4 Operator
•
2.8K Posts
0
October 21st, 2014 02:00
Hi goldcarp,
Question1:
Thin provisioning is a tool that allows an administrator to define quotas that exceed the capacity of the cluster. Doing this accomplishes two things:
1. It allows a smaller initial purchase of capacity, and the ability to simply add more as needed, promoting a capacity on demand model.
2. It allows the administrator to set larger quotas initially and not have to continually increase them as users consume their allocated capacity.
With a quota that exceeds the cluster capacity, there is nothing to stop users from consuming all available space and causing service outages fro all the users and services using the cluster.
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
Question 2:
Deduplication is applied at the subdirectory level and targets all files and directories underneath one or more root directories. As you write file to the cluster, some of those files or blocks of data in the files might be duplicates. You can run a deduplication job that scans the file system to see if the data already exits. After duplicate blocks are discovered, SmartDedupe moves a single copy of those blocks to a special set of files known as shadow stores. During this process, duplicate blocks are removed from the actual files and replaced with pointers to the shadow stores.
With post-process deduplication, new data is first stored on the storage device and then a subsequent process analyzes the data looking for commonality. This means that initial file write or modify performance is not impacted because no additional computation is required in the write path.
The deduplication job is set to run at low priority by default, so impact to your workflow should be minimal. However, it’s a good idea to wait until users have finished modifying their files on the cluster before you run the job.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Finally, please consider moving this question as-is (no need to recreate) to the proper forum for maximum visibility. Questions written to the users' own "Discussions" space don't get the same amount of attention and questions can go unanswered for a long time.
You can do so by selecting "Move" under ACTIONS along the upper-right. Then search for and select: "Isilon" which would be the most relevant for this question.
chughh
122 Posts
0
November 11th, 2014 22:00
1. Quota and Thin-Provisioning
Does SmartQuota on Isilon calculate quota based on file size or real data usage? For example, with a thin-provisioned VMware virtual disk file of 10GB capacity and 5GB usage, will the SmartQuota report 10GB or 5GB?
A1: Smartquota will take that as 5gb + overhead if included while setting up quota on the path.
2. Deduplication
Looks like this feature is added in OneFS v7.1 with SmartDedupe. Are there any impacts on VMware virtual machines from running SmartDedupe? How much space saving (percent) SmartDedupe usualy gains?
A2: Dedupe calculation is tough though dedupe scans a directory for redundant data blocks and deduplicates all redundant data stored in the directory. Available only if you activate a SmartDedupe license.
Hence there is no fixed calculation for dedupe. For more detail refer to document link attached and look for Smartquota & deduplication.
http://www.emc.com/collateral/TechnicalDocument/docu50220.pdf