Retrieving "estimated usable capacity remaining"

Brian,

I am trying to give them a ball park number based on the data that is currently on the cluster.  We had a brand new 3 node cluster and we started migrating a lot of data from Centera. These were millions of small files, probably a lot of them less than 128kb (hello triple mirror) so when i looked in IIQ it was saying that protection overhead was 44%. Much greater then 33.3% that I was assuming for 2d:1n protection.

Uhhh you and I both know that should be 33%.  What OneFS version are you running and what hardware?

InsightIQ uses the TiB etc units correctly, as far as I see.

When you compute overhead ratio, it's a division and the units cancel out -- nothing should go wrong here...

For quick checks it's nice that df on OneFS offers both unit systems:

df -h  --> power of 1024

df -H --> powers of 1000

-- Peter

With many really small files in the mix one ends up between 33% and 200% (or 3x),

so 44% still is towards the nicer end.

The example just shows that measuring the actual overhead

can make a significant difference from the nominal, configured protection level.

-- Peter

Peter,

If i take the number from IIQ and convert TiB to Terabytes I get a larger number than what i get from using the computed number.

Thank you

Do the raw capacities match? (IIQ and df)

What is your the computed protection ratio? (physical / logical)

Not that the percentage in IIQ is (physical - logical) / physical = 1 - 1/ratio, expressed as percent

Peter,

This is what i get from the API

phys_size_sum :201373832746496

log_size_sum :128663300537168

f_bavail: 24878445210

ratio :1.56512254781093

estimate in Terabytes :118.430893109624

This is what i see in InsightIQ. If i convert 119 TiB to Terabytes that's 130 Terabytes, not 118.43 as i calculated ?

cluster-1# df
Filesystem               1K-blocks         Used        Avail Capacity  Mounted on
/dev/mirror/root0          1957516       483988      1316928    27%    /
devfs                            1            1            0   100%    /dev
/dev/mirror/var0            978716       444624       455796    49%    /var
/dev/mirror/var-crash      2946284       773532      1937050    29%    /var/crash
/dev/mirror/keystore         61228           18        56312     0%    /keystore
/dev/md0                     61166         1184        55090     2%    /tmp/ufp
/dev/md1.uzip               106967       101406        -2996   103%    /base
/dev/md2                      3566          394         2888    12%    /tmp/ipmi
OneFS                 406716570960 199631935304 198866930296    50%    /ifs


cluster-1# df -h
Filesystem               Size    Used   Avail Capacity  Mounted on
/dev/mirror/root0        1.9G    473M    1.3G    27%    /
devfs                    1.0K    1.0K      0B   100%    /dev
/dev/mirror/var0         956M    434M    445M    49%    /var
/dev/mirror/var-crash    2.8G    755M    1.8G    29%    /var/crash
/dev/mirror/keystore      60M     18K     55M     0%    /keystore
/dev/md0                  60M    1.2M     54M     2%    /tmp/ufp
/dev/md1.uzip            104M     99M   -2.9M   103%    /base
/dev/md2                 3.5M    394K    2.8M    12%    /tmp/ipmi
OneFS                    379T    186T    185T    50%    /ifs


cluster1-1# df -H
Filesystem               Size    Used   Avail Capacity  Mounted on
/dev/mirror/root0        2.0G    496M    1.3G    27%    /
devfs                    1.0k    1.0k      0B   100%    /dev
/dev/mirror/var0         1.0G    455M    467M    49%    /var
/dev/mirror/var-crash    3.0G    792M    2.0G    29%    /var/crash
/dev/mirror/keystore      63M     18k     58M     0%    /keystore
/dev/md0                  63M    1.2M     56M     2%    /tmp/ufp
/dev/md1.uzip            110M    104M   -3.1M   103%    /base
/dev/md2                 3.7M    403k    3.0M    12%    /tmp/ipmi
OneFS                    416T    204T    204T    50%    /ifs

f_bavail = 24878445210 (blocksize 8 * 1024 B) => 203804223160320 Bytes => 203.8 TB => 185.4 TiB

This is consistent with both the df outputs, and basically also with IIQ's "remaining 186 TiB".

The latter could be from a slightly different pointing time, or could have undergone some unlucky rounding.

Dividing by ratio 1.56512254781093 gives: 130.2 TB or 118.5 TiB usable capacity:

again consistent with IIQs final 119 TiB figure.

What was the math for your "estimate in Terabytes :118.430893109624"?

-- Peter

$ratio = $phys_size_sum / $log_size_sum;

$fs_free_estimate = ($f_bavail * 8192 / $ratio) / 1024 / 1024 / 1024 /1024;

203804223160320 Bytes / 1.56512254781093 ratio  , how are you getting 130.2 TB out of that ?

Just divide... make sure you calculator can deal with very long integers or mantissae.

(Google-Search's builtin works, as does UNIX/Linux 'dc')

Double checking, looking only at the leading digits with any calculator:

  203.8 / 1.565 = 130.22  TB (power-of-10)

Converting to power-of-2 units:

  130.22 TB / (1.024^4) = 130.22 / 1.0995  = 118.44 TiB

I think you got it right, just that your result is already in TiB, not in decimal "terabytes".

-- Peter

sorry Peter if i am being dense here:

203804223160320 Bytes / 1.56512254781093 = 130216144061928.1 Bytes.  Now if convert that number to Terabytes I get 118.43089311 TB  ( 130216144061928.1 / 1024 / 1024 / 1024 / 1024).   What am i doing wrong, why do i end up with 118 TB and you end up with 130TB ?

Going from 130216144061928.1 Bytes. to power-of-10 TBs involves no 1024s at all,

just shifting the decimal point...  => 130.2 TB aka (decimal) terabytes in todays speak.

Sadly we have been using power-of-2 "terabytes" for so long,

misusing the decimal prefix.

But here I use terabyte and TB strictly for the decimal unit,

and tebibyte resp. TiB for power-of-2.

Bonus observation: 1 TiB = 1.1 TB within 1% accuracy

Peter,

give that my calculated number (converting bytes to Terabytes by /1024 multiple times) is identical to the number listed in IIQ, makes me wonder if the number it IIQ is really Terabytes not Tebibytes.

In our modern times, dividing by 1024 four times gives you Tebibytes, not Terabytes.

1 Tebibyte = 2^40 Bytes = 1024^4 Bytes

1 Terabyte = 10^12 Byes = 1000^4 Bytes

Problem is, we grew up calling 1024 Bytes a "Kilo"byte and so on.

As ugly as the word "Tebibyte" might appear,

calling 1024^4 Bytes a Terabyte will keep creating confusion.

IIQ correctly computes and displays Tebibytes,

and that's a free choice of the IIQ team.

Instead IIQ could use power-of-10-based Terabytes,

and the end result here would be 130.2 TB.

(try doing both exercises side by side, in two columns.)