dynamox

updated

9 years ago

D

dynamox

11 Legend

20419 Posts

87439 Points

0

7819

December 5th, 2017 19:00

Retrieving "estimated usable capacity remaining"

Hello folks,

How can i retrieve this metric either from FSA report or directly from PostgreSQL on IIQ system ? I would like to pull this value every so often via an automated process.

12-5-2017 9-46-46 PM.png

Thank you

  • dynamox

    11 Legend

    20419 Posts

    87439 Points

    1223

    0

    Posted December 8th, 2017 10:00

    Peter,

    If i take the number from IIQ and convert TiB to Terabytes I get a larger number than what i get from using the computed number.

    Thank you

  • Peter_Sero

    6 Operator

    1169 Posts

    1223

    0

    Posted December 8th, 2017 10:00

    Do the raw capacities match? (IIQ and df)

    What is your the computed protection ratio? (physical / logical)


    Not that the percentage in IIQ is (physical - logical) / physical = 1 - 1/ratio, expressed as percent

  • dynamox

    11 Legend

    20419 Posts

    87439 Points

    1226

    0

    Posted December 8th, 2017 20:00

    Peter,

    This is what i get from the API

    phys_size_sum :201373832746496

    log_size_sum :128663300537168

    f_bavail: 24878445210

    ratio :1.56512254781093

    estimate in Terabytes :118.430893109624

    This is what i see in InsightIQ. If i convert 119 TiB to Terabytes that's 130 Terabytes, not 118.43 as i calculated ?

    12-8-2017 11-44-04 PM.png

    cluster-1# df

    Filesystem               1K-blocks         Used        Avail Capacity  Mounted on

    /dev/mirror/root0          1957516       483988      1316928    27%    /

    devfs                            1            1            0   100%    /dev

    /dev/mirror/var0            978716       444624       455796    49%    /var

    /dev/mirror/var-crash      2946284       773532      1937050    29%    /var/crash

    /dev/mirror/keystore         61228           18        56312     0%    /keystore

    /dev/md0                     61166         1184        55090     2%    /tmp/ufp

    /dev/md1.uzip               106967       101406        -2996   103%    /base

    /dev/md2                      3566          394         2888    12%    /tmp/ipmi

    OneFS                 406716570960 199631935304 198866930296    50%    /ifs

    cluster-1# df -h

    Filesystem               Size    Used   Avail Capacity  Mounted on

    /dev/mirror/root0        1.9G    473M    1.3G    27%    /

    devfs                    1.0K    1.0K      0B   100%    /dev

    /dev/mirror/var0         956M    434M    445M    49%    /var

    /dev/mirror/var-crash    2.8G    755M    1.8G    29%    /var/crash

    /dev/mirror/keystore      60M     18K     55M     0%    /keystore

    /dev/md0                  60M    1.2M     54M     2%    /tmp/ufp

    /dev/md1.uzip            104M     99M   -2.9M   103%    /base

    /dev/md2                 3.5M    394K    2.8M    12%    /tmp/ipmi

    OneFS                    379T    186T    185T    50%    /ifs

    cluster1-1# df -H

    Filesystem               Size    Used   Avail Capacity  Mounted on

    /dev/mirror/root0        2.0G    496M    1.3G    27%    /

    devfs                    1.0k    1.0k      0B   100%    /dev

    /dev/mirror/var0         1.0G    455M    467M    49%    /var

    /dev/mirror/var-crash    3.0G    792M    2.0G    29%    /var/crash

    /dev/mirror/keystore      63M     18k     58M     0%    /keystore

    /dev/md0                  63M    1.2M     56M     2%    /tmp/ufp

    /dev/md1.uzip            110M    104M   -3.1M   103%    /base

    /dev/md2                 3.7M    403k    3.0M    12%    /tmp/ipmi

    OneFS                    416T    204T    204T    50%    /ifs

  • Peter_Sero

    6 Operator

    1169 Posts

    1226

    0

    Posted December 11th, 2017 14:00

    f_bavail = 24878445210 (blocksize 8 * 1024 B) => 203804223160320 Bytes => 203.8 TB => 185.4 TiB

    This is consistent with both the df outputs, and basically also with IIQ's "remaining 186 TiB".

    The latter could be from a slightly different pointing time, or could have undergone some unlucky rounding.

    Dividing by ratio 1.56512254781093 gives: 130.2 TB or 118.5 TiB usable capacity:

    again consistent with IIQs final 119 TiB figure.

    What was the math for your "estimate in Terabytes :118.430893109624"?

    -- Peter

  • dynamox

    11 Legend

    20419 Posts

    87439 Points

    1226

    0

    Posted December 11th, 2017 21:00

    $ratio = $phys_size_sum / $log_size_sum;

    $fs_free_estimate = ($f_bavail * 8192 / $ratio) / 1024 / 1024 / 1024 /1024;

    203804223160320 Bytes / 1.56512254781093 ratio  , how are you getting 130.2 TB out of that ?

  • Peter_Sero

    6 Operator

    1169 Posts

    1226

    0

    Posted December 12th, 2017 08:00

    Just divide... make sure you calculator can deal with very long integers or mantissae.

    (Google-Search's builtin works, as does UNIX/Linux 'dc')

    Double checking, looking only at the leading digits with any calculator:

      203.8 / 1.565 = 130.22  TB (power-of-10)

    Converting to power-of-2 units:

      130.22 TB / (1.024^4) = 130.22 / 1.0995  = 118.44 TiB

    I think you got it right, just that your result is already in TiB, not in decimal "terabytes".

    -- Peter

  • dynamox

    11 Legend

    20419 Posts

    87439 Points

    1226

    0

    Posted December 12th, 2017 10:00

    sorry Peter if i am being dense here:

    203804223160320 Bytes / 1.56512254781093 = 130216144061928.1 Bytes.  Now if convert that number to Terabytes I get 118.43089311 TB  ( 130216144061928.1 / 1024 / 1024 / 1024 / 1024).   What am i doing wrong, why do i end up with 118 TB and you end up with 130TB ?



  • Peter_Sero

    6 Operator

    1169 Posts

    1226

    0

    Posted December 12th, 2017 11:00

    Going from 130216144061928.1 Bytes. to power-of-10 TBs involves no 1024s at all,

    just shifting the decimal point...  => 130.2 TB aka (decimal) terabytes in todays speak.

    Sadly we have been using power-of-2 "terabytes" for so long,

    misusing the decimal prefix.

    But here I use terabyte and TB strictly for the decimal unit,

    and tebibyte resp. TiB for power-of-2.

    Bonus observation: 1 TiB = 1.1 TB within 1% accuracy

  • dynamox

    11 Legend

    20419 Posts

    87439 Points

    1226

    0

    Posted December 14th, 2017 10:00

    Peter,

    give that my calculated number (converting bytes to Terabytes by /1024 multiple times) is identical to the number listed in IIQ, makes me wonder if the number it IIQ is really Terabytes not Tebibytes.

  • Peter_Sero

    6 Operator

    1169 Posts

    765

    0

    Posted December 14th, 2017 10:00

    In our modern times, dividing by 1024 four times gives you Tebibytes, not Terabytes.

    1 Tebibyte = 2^40 Bytes = 1024^4 Bytes

    1 Terabyte = 10^12 Byes = 1000^4 Bytes

    Problem is, we grew up calling 1024 Bytes a "Kilo"byte and so on.

    As ugly as the word "Tebibyte" might appear,

    calling 1024^4 Bytes a Terabyte will keep creating confusion.

    IIQ correctly computes and displays Tebibytes,

    and that's a free choice of the IIQ team.

    Instead IIQ could use power-of-10-based Terabytes,

    and the end result here would be 130.2 TB.

    (try doing both exercises side by side, in two columns.)