Highlighted
Octavius1
1 Nickel

Occupation excessive when copy files to ISILON

The original information occupied 14GB and after migrating to Linux (cp -a) occupies 36GB. Anyone have information about it?

Thank you !!!

0 Kudos
2 Replies

Re: Occupation excessive when copy files to ISILON

First, about determinate the amount of space usage on OneFS, take a look in the KB 16635 support.emc.com/kb/16635

(..)

To view the space used to store data in a specified directory, run the following command replacing  with the path to the directory that you want to <path>

measure:

du -sSHAk /ifs/<path>

For example, the command and output for the  directory is: /ifs/data

du -sSHAk /ifs/data

71913088 /ifs/data

To view the space used to store and protect data in a specified directory, run the following command replacing  with the path to the directory that you <path>

want to measure:

du -sSHk /ifs/<path>

For example, the command and output for the  directory is: /ifs/data

du -sSHk /ifs/data

110838453 /ifs/data

(..)

Second, the amount of require space for parity is based on number of nodes in the cluster, size of files and protection level configured (default level is N+2:1).

To have a start point, take a look in the OneFS Administration Guide, Chapter 14 -  Data layout with FlexProtect and in the paper docu42429_White-Paper_-High-Availability-and-Data-Protection-with-EMC-isilon-Scale-Out-NAS.

Pay attention that files with smaller size then 128 KB are mirrored 2x. So, a directory with a huge amount of small files, can use a considerable amount of space to protect data.

coryminton
1 Nickel

Re: Occupation excessive when copy files to ISILON

Octavius,

Out of curiosity, where are you seeing this metrics you reference in terms of space consumed?  Was it on the Isilon GUI or when viewing the share through an application or Windows explorer?  I ask because if this size on disk is being viewed through some windows app, it may not be accurate as Microsoft has stated on their support forums that this metric can be calculated incorrectly one shared storage or NAS and I have personally experienced this many times.

Folder 'Size on Disk' Incorrect - Microsoft Community

The easiest work around is to create 16TB hard quotas for that directory, if reporting size on disk is important to an application using this metric.

If it's showing this greater consumption in Isilon OneFS or InsightIQ, then it is probably due to a large number of files being under 128k in size, which will not stripe efficiently on Isilon.  If the consumption is a problem, I would consider going to 2X mirroring as the data protection policy for that particular share/directory and increase data protection through snapshots and replication to appropriately protect that data.

Hopefully this is helpful.