File size, parity overhead and used space

The OneFS filesystem explorer show a bogus disk usage, it accounts only for rounding up to 8KB blocks but ignores the more important protection overhead.

For the larger file  in your example, note that protection is added to *each* stripe,

so there should be 2 x (5 + 2) = 14 stripe units on disk.

I can't comment on what Windows is reporting. Better start with "du" right on the Isilon cluster for reference, and don't expect things to be too simple when file sizes grow from KB to some MB. To illustrate this, run this small script (it creates a test.dat file in the local folder which should be under /ifs), and plot the results:

=========

du-test.zsh

=========

#!/bin/zsh

for i in {1..100}

do

  (( sizeKB = 32 * i ))

  dd if=/dev/zero of=test.dat bs=${sizeKB}k count=1 2>/dev/null

  sync

  echo $(du -A -k test.dat) $(du -k test.dat) |

  awk ' { fileKB=$1; diskKB=$3;

         if (fileKB==0) { print "Error: File size 0KB, exiting"; exit; }

         ratio=diskKB/fileKB;

         overhd=100*(ratio-1);

         # graphval=int(diskKB/32);

         graphval=int(overhd/3);

         printf "%5dKB file, %5dKB disk, %6.3f ratio, +%5.1f%% overhd, graph: |%*.*s* \n",

                fileKB, diskKB, ratio, overhd, graphval, graphval, ""; }'

done

rm -f test.dat

========

For your convenience the script already makes a (lousy) attempt at plotting the overhead percentage -- which "basically" drops from around +200% down towards +50% on my 3 node pool and +2n:1 protection. See (below) how bumpy the curve is? Each bump indicates another stripe, and I think the chart explains more than an exact formula... The tables give the asymptotic values for very large files.

Cheers

-- Peter

Hi Peter,

thank you for the hints.

But one think I still don't understand:

For the larger file  in your example, note that protection is added to *each* stripe, so there should be 2 x (5 + 2) = 14 stripe units on disk.

I have given two parity strip units at each stripe in the second table above - but following your explanation - why do I need total 10 data stripes although eight are enough to set up the total file of 1024 KB?

I tried it again with a 1 Byte File. For that file Windows shows the same storage size as for a 64 or 128 KB file:

• Size: 1 Bytes (1 Bytes)
• Size at the Medium: 404 KB (413.696 Bytes)

I will do the test with your script.

EDIT: I run your script. Here is the output:

My manually calculated file sizes are not the same, but really similar. I suppose the publishes "protection overhead" table of EMC was done with big files. With smaller files this values will not be reached.

> I have given two parity strip units at each stripe in the second table above - but following your explanation - why do I need total 10 data stripes although eight are enough to set up the total file of 1024 KB?

Small stripes a built from 3 replicated units.

From a certain size on, the full stripe width for the pool is used, even when the stripe units have not yet reached 128K each.  With more data being written to the file the units get uniformly filled up. That gives a different picture from what you might have in mind, like adding one full stripe unit at a time while growing (which would lead to repeatedly re-striping the full active stripe).

That was the nutshell version; as Rob suggested, becoming familiar with erase coding helps...

Cheers

-- Peter