14

I would like to convert the ZFS output of "10.9T" to actual bytes, using something in a line or two, rather than run generic math functions, and if conditions for T, G, M, etc.. Is there an efficient way to do this?

For now, I have something like this:

MINFREE="50G"
POOLSIZE=`zpool list $POOLNAME -o size` #Size 10.9T
POOLSIZE=$(echo "$POOLSIZE" | grep -e [[:digit:))]  #10.9T
POOLFREE=500M #as an example
let p=POOLSIZE x=POOLFREE y=MINFREE z=POOLSIZE; 
CALC=$(expr "echo $((x / y))")


if [ "${CALC}" < 1 ]; then
  # we are less than our min free space
  echo alert
fi

This produces an error: can't run the expression on 10.9T, or 50G because they arent numbers.

Is there a known bash function for this?

I also like the convenience of specifying it like i did there in the MINFREE var at the top. So an easy way to convert would be nice.

This is what I was hoping to avoid (making case for each letter), the script looks clean though.

Edit: Thanks for all the comments! Here is the code I have now. , relevant parts atleast;

POOLNAME=san
INFORMAT=auto
#tip; specify in Gi, Ti, etc.. (optional)
MINFREE=500Gi
OUTFORMAT=iec
NOW=`date`;
LOGPATH=/var/log/zfs/zcheck.log
BOLD=$(tput bold)
BRED=${txtbld}$(tput setaf 1)
BGREEN=${txtbld}$(tput setaf 2)
BYELLOW=${txtbld}$(tput setaf 3)
TXTRESET=$(tput sgr0);

# ZFS Freespace check
#poolsize, how large is it
POOLSIZE=$(zpool list $POOLNAME -o size -p)
POOLSIZE=$(echo "$POOLSIZE" | grep -e [[:digit:]])
POOLSIZE=$(numfmt --from=iec $POOLSIZE)
#echo poolsize $POOLSIZE

#poolfree, how much free space left
POOLFREE=`zpool list $POOLNAME -o free`
#POOLFREE=$(echo "$POOLFREE" | grep -e [[:digit:]]*.[[:digit:]].)
POOLFREE=$(echo "$POOLFREE" | grep -e [[:digit:]])
POOLFREE=$(numfmt --from=$INFORMAT $POOLFREE)
#echo poolfree $POOLFREE
#grep -e "vault..[[:digit:]]*.[[:digit:]].")

#minfree, how low can we go, before alerting
MINFREE=$(numfmt --from=iec-i $MINFREE)
#echo minfree $MINFREE


#FORMATTED DATA USED FOR DISPLAYING THINGS
#echo formattiing sizes:
F_POOLSIZE=$(numfmt --from=$INFORMAT --to=$OUTFORMAT  $POOLSIZE)
F_POOLFREE=$(numfmt --from=$INFORMAT --to=$OUTFORMAT $POOLFREE)
F_MINFREE=$(numfmt --from=$INFORMAT --to=$OUTFORMAT $MINFREE)
F_MINFREE=$(numfmt --from=$INFORMAT --to=$OUTFORMAT $MINFREE)
#echo
printf "${BGREEN}$F_POOLSIZE - current pool size"
printf "\n$F_MINFREE - mininium freespace allowed/as specified"

# OPERATE/CALCULATE SPACE TEST
#echo ... calculating specs, please wait..
#let f=$POOLFREE m=$MINFREE x=m/f;
declare -i x=$POOLFREE/$MINFREE;
# will be 0 if has reached low threshold, if poolfree/minfree
#echo $x
#IF_CALC=$(numfmt --to=iec-i $CALC)
if ! [ "${x}" == 1 ]; then
  #printf "\n${BRED}ALERT! POOL FREESPACE is low! ($F_POOLFREE)"
  printf "\n${BRED}$F_POOLFREE ${BYELLOW}- current freespace! ${BRED}(ALERT!}${BYELLOW} Is below your preset threshold!";
  echo
else
  printf "\nPOOLFREE - ${BGREEN}$F_POOLFREE${TXTRESET}- current freespace";
  #sleep 3
fi
6
  • 15
    Why don't you just use -p so you can get the actual numbers and skip most or all of this? Jun 2, 2021 at 13:59
  • 3
    if [ "${CALC}" < 1 ]; then probably doesn't do what you want, the < there is an input redirection. It should give an error unless you have a file called 1.
    – ilkkachu
    Jun 3, 2021 at 9:29
  • 5
    So, is 50G gibibyte or gigabyte?
    – MonkeyZeus
    Jun 3, 2021 at 12:24
  • 2
    While you're at it, also consider using -H to remove the text header, leaving only the specific field you asked for, and eliminating the need to grep or write goofy regexes. Jun 3, 2021 at 23:45
  • 1
    Isn't it generally undesirable to reverse engineer output that was carefully formatted to be human readable, i.e. for things other than one-offs, shouldn't one use programmatic facilities the OS provides, like the proc filesystem? I understand from your comment that this happens to be your use case; but most people facing this problem should take a step back, take in the larger picture and find a way to avoid that. Jun 4, 2021 at 0:36

4 Answers 4

41

There is no good way to convert zfs's human-readable numbers to actual bytes. The human-readable numbers are rounded off and therefore inexact.

If you want exact numbers, use the -p option (machine parseable), and the output will be in bytes, which you can parse and format however you wish.

$ zfs list tank/var; zfs list -p tank/var
NAME       USED  AVAIL     REFER  MOUNTPOINT
tank/var  8.33G   387G     6.90G  /var
NAME            USED         AVAIL       REFER  MOUNTPOINT
tank/var  8948584448  415137447936  7407120384  /var

But parsing zfs's human-readable output and converting to "exact" numbers is not possible. Since the human readable numbers are only specified to (say) three significant figures, your "exact" extrapolation will also be accurate only to three figures.

TiB=$((2**40))
GiB=$((2**30))

# MINFREE=$((50*$TiB)) # 50 TiB
MINFREE=$((50*$GiB))   # 50 GiB

POOLFREE=$(zpool list -Hpo free "$POOLNAME") #Free in bytes

if [ "$POOLFREE" -lt "$MINFREE" ]; then
  printf "alert\n"
else
  printf "no alert -- %d bytes free >= %d byte minimum\n" "$POOLFREE" "$MINFREE"
fi
5
  • This may be helpful! Thanks for pointing that out. Jun 1, 2021 at 21:16
  • 4
    This! Ask the program generating the output to generate it in the form you need. Jun 2, 2021 at 9:54
  • Is there also an option to suppress the header line? Jun 2, 2021 at 15:38
  • 2
    2**30 is GiB. 10**3 is GB.
    – l0b0
    Jun 2, 2021 at 21:59
  • IOW, the answer to "What’s an efficient way to convert human-readable sizes to byte quantities?" is, "let the computer's existing software do it for you."
    – RonJohn
    Jun 4, 2021 at 5:24
32

You can use numfmt (in Debian and derivatives it is part of coreutils so it should be there already):

numfmt - Convert numbers from/to human-readable strings

$ numfmt --from=iec-i 50.1Gi
53794465383

it can also read the value from stdin

$ echo "50.1Gi" | numfmt --from=iec-i
53794465383

Be careful, it takes into account the locale for the decimal separator.

1
  • 1
    #TIL numfmt. It's amazing that after years of trawling through coreutils manpages, there are still gems like this hidden in plain sight that I never saw before today! Jun 3, 2021 at 15:20
9

zpool list can provide the numbers in bytes. e.g. listing three pools (15T, 29T, and 416G) on my main zfs server.

First, without -H and -p:

$ zpool list -o name,size,alloc,free,capacity
NAME     SIZE  ALLOC   FREE    CAP
backup  14.5T  6.15T  8.40T    42%
export    29T  17.8T  11.2T    61%
ganesh   416G   169G   247G    40%

And again, with -H and -p

$ zpool list -H -p -o name,size,alloc,free,capacity
backup  15994458210304  6763872280576   9230585929728   42
export  31885837205504  19592775573504  12293061632000  61
ganesh  446676598784    181604904960    265071693824    40

The output is tab separated, so is easily processed with awk or cut or whatever you like (even a shell while read loop if you insist). The capacity field is the percentage used, so is particularly useful if you want to email an alert if a pool gets below 10% or 20% free.

  • -H is Scripted mode. Do not display headers, and separate fields by a single tab instead of arbitrary space.
  • -p prints "parseable" as opposed to "human readable" format (i.e. bytes)

BTW, recent versions of ZFS have the zfs and zpool man pages split into separate pages for their sub-commands, properties, concepts, etc. If that's what you're running, see the man pages for zpool-list and zpoolprops for more details. Otherwise, just man zpool.

4

As it is tagged and worded with bash I throw this in for fun.

No validation. (I.e. if unit exists etc.)

SI base quantity 1000 (10^3):

#!/bin/bash

declare -A s=([Y]=24 [Z]=21 [E]=18 [P]=15 [T]=12 [G]=9 [M]=6 [K]=3)

input="$1"
suffix="${input: -1}"
number="${input:0: -1}"

printf "%.0f bytes\n" "${number}e+${s[$suffix]}"
$ ./bashsize10 1.6T
1600000000000 bytes

$ ./bashsize10 3681.914Y
3681914000000000000130023424 bytes

IEC base quantity 1024 (2^10) (using 2 decimal)

Max 81.914P (64-bit) (8.191 if using 3 decimals etc.)

#!/bin/bash

declare -A s=([P]=50 [T]=40 [G]=30 [M]=20 [K]=10)

input="$1"
suffix="${input: -1}"
number="${input:0: -1}"
d=$(printf "%.0f" "${number}e+2")

printf "%d bytes\n" "$(( d * 2 ** s["$suffix"] / 100 ))"
$ ./bashsize2 1.6T
1759218604441 bytes
2
  • i like the simple array declared, so that actually puts the problem such as the linked one, resolved, into one line, more like i was lookin for originally. i'm giving the point for that also, because i ended up using declare -i x=$POOLFREE/$MINFREE; will be 0 if low, 1 if high, when using straight numbers taken from @Eduardo Trápani solution . Jun 2, 2021 at 0:33
  • 2
    @BrianThomas: Yes. numfmt is a nice tool. As I'm sure you know, bash can not do floating point arithmetic. printf, however, can handle e-notation. Find it fun to find ways of circumvent the limitations, though usually for fun - like the bash variant at end of this one: unix.stackexchange.com/a/651945/140633 :P
    – ibuprofen
    Jun 2, 2021 at 1:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.