Unsolved
This post is more than 5 years old
7 Posts
0
2420
Occupation excessive when copy files to ISILON
The original information occupied 14GB and after migrating to Linux (cp -a) occupies 36GB. Anyone have information about it?
Thank you !!!
Unsolved
This post is more than 5 years old
7 Posts
0
2420
The original information occupied 14GB and after migrating to Linux (cp -a) occupies 36GB. Anyone have information about it?
Thank you !!!
Top
arthur.r.olivei
11 Posts
1
October 14th, 2014 07:00
First, about determinate the amount of space usage on OneFS, take a look in the KB 16635 support.emc.com/kb/16635
(..)
To view the space used to store data in a specified directory, run the following command replacing with the path to the directory that you want to
measure:
du -sSHAk /ifs/
For example, the command and output for the directory is: /ifs/data
du -sSHAk /ifs/data
71913088 /ifs/data
To view the space used to store and protect data in a specified directory, run the following command replacing with the path to the directory that you
want to measure:
du -sSHk /ifs/
For example, the command and output for the directory is: /ifs/data
du -sSHk /ifs/data
110838453 /ifs/data
(..)
Second, the amount of require space for parity is based on number of nodes in the cluster, size of files and protection level configured (default level is N+2:1).
To have a start point, take a look in the OneFS Administration Guide, Chapter 14 - Data layout with FlexProtect and in the paper docu42429_White-Paper_-High-Availability-and-Data-Protection-with-EMC-isilon-Scale-Out-NAS.
Pay attention that files with smaller size then 128 KB are mirrored 2x. So, a directory with a huge amount of small files, can use a considerable amount of space to protect data.
coryminton
21 Posts
1
October 22nd, 2014 15:00
Octavius,
Out of curiosity, where are you seeing this metrics you reference in terms of space consumed? Was it on the Isilon GUI or when viewing the share through an application or Windows explorer? I ask because if this size on disk is being viewed through some windows app, it may not be accurate as Microsoft has stated on their support forums that this metric can be calculated incorrectly one shared storage or NAS and I have personally experienced this many times.
Folder 'Size on Disk' Incorrect - Microsoft Community
The easiest work around is to create 16TB hard quotas for that directory, if reporting size on disk is important to an application using this metric.
If it's showing this greater consumption in Isilon OneFS or InsightIQ, then it is probably due to a large number of files being under 128k in size, which will not stripe efficiently on Isilon. If the consumption is a problem, I would consider going to 2X mirroring as the data protection policy for that particular share/directory and increase data protection through snapshots and replication to appropriately protect that data.
Hopefully this is helpful.