2

I'm currently working on a project called 'dmCLI' which means 'download manager command line interface'. I'm using curl to multi-part download a file. And I'm having trouble with my code. I can't convert string to int.

Here's my full code. I also uploaded my code into Github Repo. Here it is: dmCLI GitHub Repository

#!/bin/bash
RED='\033[0;31m'
NC='\033[0m'

help() {
    echo "dmcli [ FILENAME:path ] [ URL:uri ]"
}

echo -e "author: ${RED}@atahabaki${NC}"
echo -e "     __      _______   ____"
echo -e " ___/ /_ _  / ___/ /  /  _/"
echo -e "/ _  /  ' \/ /__/ /___/ /  "
echo -e "\_,_/_/_/_/\___/____/___/  "
echo -e "                           "
echo -e "${RED}Downloading${NC} has never been ${RED}easier${NC}.\n"

if [ $# == 2 ]
then
    filename=$1;
    url=$2
    headers=`curl -I ${url} > headers`
    contentlength=`cat headers | grep -E '[Cc]ontent-[Ll]ength:' | sed 's/[Cc]ontent-[Ll]ength:[ ]*//g'`
    acceptranges=`cat headers | grep -E '[Aa]ccept-[Rr]anges:' | sed 's/[Aa]ccept-[Rr]anges:[ ]*//g'`
    echo -e '\n'
    if [ "$acceptranges" = "bytes" ]
    then 
        echo File does not allow multi-part download.   
    else
        echo File allows multi-part download.
    fi
    echo "$(($contentlength + 9))"
    # if [acceptranges == 'bytes']
    # then
    # divisionresult = $((contentlength/9))
    # use for to create ranges...
else 
    help
fi

# First get Content-Length via regex or request,
# Then Create Sequences,
# After Start Downloading,
# Finally, re-assemble to 1 file.

I want to divide contentlength's value by 9. I tried this:

echo "$(($contentlength/9))"

It's getting below error:

/9")syntax error: invalid arithmetic operator (error token is "

I'm using localhost written in node.js. I added head responses. It returns Content-Length of the requested file, and the above dmCLI.sh gets correctly the Content-Length value.

./dmcli.sh home.html http://127.0.0.1:404/

A HEAD request to http://127.0.0.1:404/ returns: Content-Length: 283, Accept-Range: bytes

The dmCLI works for getting the value, but when I want to access its value, it won't work.

Simple actions work like:

echo "$contentlength"

But I can't access it by using this:

echo "$contentlength bytes"

and here:

echo "$(($contentlength + 9))"

returns 9 but I'm expecting 292. Where's the problem; why is it not working?

  • If echo "$contentlength" works, there is no reason echo "$contentlength bytes" does not work as well. – Weijun Zhou Mar 14 at 9:32
  • 2
    Could be the value of the variable ends in a CR character (line delimiters in HTTP headers are CRLF), 12345\r bytes would show up as " bytes" in a terminal as CR moves the cursor to the beginning of the line. – Stéphane Chazelas Mar 14 at 9:34
  • @StéphaneChazelas You're right. CR does not have this behavior when redirected. One can redirect the output and examine the result. – Weijun Zhou Mar 14 at 9:36
  • echo " $contentlength bytes" it returns this: ``` bytes``` – A. Taha Baki Mar 14 at 9:38
  • The way it's written (not proper header parsing, no sanitisation of the content length), this script has an arbitrary command injection vulnerability. Do not use a shell for this kind of thing. perl/python have HTTP or libcurl APIs. – Stéphane Chazelas Mar 14 at 9:38
2

The regex below will extract the number of bytes, only the number:

contentlength=$(
  LC_ALL=C sed '
    /^[Cc]ontent-[Ll]ength:[[:blank:]]*0*\([0-9]\{1,\}\).*$/!d
    s//\1/; q' headers
)

After the above change, the contentlength variable will be only made of decimal digits (with leading 0s removes so the shell doesn't consider the number as octal), thus the 2 lines below will display the same result:

echo "$(($contentlength/9))"

echo "$((contentlength/9))"
  • It worked but what's the difference between echo $((contentlength/9)) and echo $(($contentlength))? – A. Taha Baki Mar 14 at 9:51
  • I won't call $(($contentlength/9)) incorrect, just that the $ is unnecessary, but there are cases where this $ is needed. – Weijun Zhou Mar 14 at 9:57
  • There will be ARITHMETIC EVALUATION with this syntax, you can read this in the man bash: Within an expression, shell variables may also be referenced by name without using the parameter expansion syntax.. In other words: do not use $ for variable with this syntax. – Jay jargot Mar 14 at 9:58
  • 1
    But what if parameter expansion is needed? $ in $((${b/5/7}+1)) is needed. – Weijun Zhou Mar 14 at 10:00
  • 1
    @Jayjargot, there will be arithmetic evaluation with both syntax in ksh, bash and zsh (hence the command injection vulnerabilities when the input is not sanitized), one difference is when the value is like 9+9. Then contentlength/9 will give 2 and $contentlength/9 will give 1. – Stéphane Chazelas Mar 14 at 10:01
0
headers=`curl -I ${url} > headers`
contentlength=`cat headers | grep -E '[Cc]ontent-[Ll]ength:' | sed 's/[Cc]ontent-[Ll]ength:[ ]*//g'`
echo "$(($contentlength/9))"

It's getting below error:

/9")syntax error: invalid arithmetic operator (error token is "

HTTP headers always end with CR/LF, not just LF; your `cat headers | ...` command expansion will remove the ending LF, but leave the CR alone, which will cause that strange error.

% var=`printf "%s\r\n" 333`
% echo var={$var}
}ar={333
% echo "$((var / 3))"
")syntax error: invalid arithmetic operator (error token is "

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.