What’s an efficient way to convert human-readable sizes to byte quantities?

Question

I would like to convert the ZFS output of "10.9T" to actual bytes, using something in a line or two, rather than run generic math functions, and if conditions for T, G, M, etc.. Is there an efficient way to do this?

For now, I have something like this:

MINFREE="50G"
POOLSIZE=`zpool list $POOLNAME -o size` #Size 10.9T
POOLSIZE=$(echo "$POOLSIZE" | grep -e [[:digit:))]  #10.9T
POOLFREE=500M #as an example
let p=POOLSIZE x=POOLFREE y=MINFREE z=POOLSIZE; 
CALC=$(expr "echo $((x / y))")


if [ "${CALC}" < 1 ]; then
  # we are less than our min free space
  echo alert
fi

This produces an error: can't run the expression on 10.9T, or 50G because they arent numbers.

Is there a known bash function for this?

I also like the convenience of specifying it like i did there in the MINFREE var at the top. So an easy way to convert would be nice.

This is what I was hoping to avoid (making case for each letter), the script looks clean though.

Edit: Thanks for all the comments! Here is the code I have now. , relevant parts atleast;

POOLNAME=san
INFORMAT=auto
#tip; specify in Gi, Ti, etc.. (optional)
MINFREE=500Gi
OUTFORMAT=iec
NOW=`date`;
LOGPATH=/var/log/zfs/zcheck.log
BOLD=$(tput bold)
BRED=${txtbld}$(tput setaf 1)
BGREEN=${txtbld}$(tput setaf 2)
BYELLOW=${txtbld}$(tput setaf 3)
TXTRESET=$(tput sgr0);

# ZFS Freespace check
#poolsize, how large is it
POOLSIZE=$(zpool list $POOLNAME -o size -p)
POOLSIZE=$(echo "$POOLSIZE" | grep -e [[:digit:]])
POOLSIZE=$(numfmt --from=iec $POOLSIZE)
#echo poolsize $POOLSIZE

#poolfree, how much free space left
POOLFREE=`zpool list $POOLNAME -o free`
#POOLFREE=$(echo "$POOLFREE" | grep -e [[:digit:]]*.[[:digit:]].)
POOLFREE=$(echo "$POOLFREE" | grep -e [[:digit:]])
POOLFREE=$(numfmt --from=$INFORMAT $POOLFREE)
#echo poolfree $POOLFREE
#grep -e "vault..[[:digit:]]*.[[:digit:]].")

#minfree, how low can we go, before alerting
MINFREE=$(numfmt --from=iec-i $MINFREE)
#echo minfree $MINFREE


#FORMATTED DATA USED FOR DISPLAYING THINGS
#echo formattiing sizes:
F_POOLSIZE=$(numfmt --from=$INFORMAT --to=$OUTFORMAT  $POOLSIZE)
F_POOLFREE=$(numfmt --from=$INFORMAT --to=$OUTFORMAT $POOLFREE)
F_MINFREE=$(numfmt --from=$INFORMAT --to=$OUTFORMAT $MINFREE)
F_MINFREE=$(numfmt --from=$INFORMAT --to=$OUTFORMAT $MINFREE)
#echo
printf "${BGREEN}$F_POOLSIZE - current pool size"
printf "\n$F_MINFREE - mininium freespace allowed/as specified"

# OPERATE/CALCULATE SPACE TEST
#echo ... calculating specs, please wait..
#let f=$POOLFREE m=$MINFREE x=m/f;
declare -i x=$POOLFREE/$MINFREE;
# will be 0 if has reached low threshold, if poolfree/minfree
#echo $x
#IF_CALC=$(numfmt --to=iec-i $CALC)
if ! [ "${x}" == 1 ]; then
  #printf "\n${BRED}ALERT! POOL FREESPACE is low! ($F_POOLFREE)"
  printf "\n${BRED}$F_POOLFREE ${BYELLOW}- current freespace! ${BRED}(ALERT!}${BYELLOW} Is below your preset threshold!";
  echo
else
  printf "\nPOOLFREE - ${BGREEN}$F_POOLFREE${TXTRESET}- current freespace";
  #sleep 3
fi

Answer 1

There is no good way to convert zfs's human-readable numbers to actual bytes. The human-readable numbers are rounded off and therefore inexact.

If you want exact numbers, use the -p option (machine parseable), and the output will be in bytes, which you can parse and format however you wish.

$ zfs list tank/var; zfs list -p tank/var
NAME       USED  AVAIL     REFER  MOUNTPOINT
tank/var  8.33G   387G     6.90G  /var
NAME            USED         AVAIL       REFER  MOUNTPOINT
tank/var  8948584448  415137447936  7407120384  /var

But parsing zfs's human-readable output and converting to "exact" numbers is not possible. Since the human readable numbers are only specified to (say) three significant figures, your "exact" extrapolation will also be accurate only to three figures.

TiB=$((2**40))
GiB=$((2**30))

# MINFREE=$((50*$TiB)) # 50 TiB
MINFREE=$((50*$GiB))   # 50 GiB

POOLFREE=$(zpool list -Hpo free "$POOLNAME") #Free in bytes

if [ "$POOLFREE" -lt "$MINFREE" ]; then
  printf "alert\n"
else
  printf "no alert -- %d bytes free >= %d byte minimum\n" "$POOLFREE" "$MINFREE"
fi

Answer 2

You can use numfmt (in Debian and derivatives it is part of coreutils so it should be there already):

numfmt - Convert numbers from/to human-readable strings

$ numfmt --from=iec-i 50.1Gi
53794465383

it can also read the value from stdin

$ echo "50.1Gi" | numfmt --from=iec-i
53794465383

Be careful, it takes into account the locale for the decimal separator.

Answer 3

zpool list can provide the numbers in bytes. e.g. listing three pools (15T, 29T, and 416G) on my main zfs server.

First, without -H and -p:

$ zpool list -o name,size,alloc,free,capacity
NAME     SIZE  ALLOC   FREE    CAP
backup  14.5T  6.15T  8.40T    42%
export    29T  17.8T  11.2T    61%
ganesh   416G   169G   247G    40%

And again, with -H and -p

$ zpool list -H -p -o name,size,alloc,free,capacity
backup  15994458210304  6763872280576   9230585929728   42
export  31885837205504  19592775573504  12293061632000  61
ganesh  446676598784    181604904960    265071693824    40

The output is tab separated, so is easily processed with awk or cut or whatever you like (even a shell while read loop if you insist). The capacity field is the percentage used, so is particularly useful if you want to email an alert if a pool gets below 10% or 20% free.

• -H is Scripted mode. Do not display headers, and separate fields by a single tab instead of arbitrary space.
• -p prints "parseable" as opposed to "human readable" format (i.e. bytes)

BTW, recent versions of ZFS have the zfs and zpool man pages split into separate pages for their sub-commands, properties, concepts, etc. If that's what you're running, see the man pages for zpool-list and zpoolprops for more details. Otherwise, just man zpool.

Answer 4

As it is tagged and worded with bash I throw this in for fun.

No validation. (I.e. if unit exists etc.)

SI base quantity 1000 (10^3):

#!/bin/bash

declare -A s=([Y]=24 [Z]=21 [E]=18 [P]=15 [T]=12 [G]=9 [M]=6 [K]=3)

input="$1"
suffix="${input: -1}"
number="${input:0: -1}"

printf "%.0f bytes\n" "${number}e+${s[$suffix]}"

$ ./bashsize10 1.6T
1600000000000 bytes

$ ./bashsize10 3681.914Y
3681914000000000000130023424 bytes

IEC base quantity 1024 (2^10) (using 2 decimal)

Max 81.914P (64-bit) (8.191 if using 3 decimals etc.)

#!/bin/bash

declare -A s=([P]=50 [T]=40 [G]=30 [M]=20 [K]=10)

input="$1"
suffix="${input: -1}"
number="${input:0: -1}"
d=$(printf "%.0f" "${number}e+2")

printf "%d bytes\n" "$(( d * 2 ** s["$suffix"] / 100 ))"

$ ./bashsize2 1.6T
1759218604441 bytes