Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm writing a bash script that extensively uses wget. To define all common parameters in one place I store them on variables. Here's a piece of code:

useragent='--user-agent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0"'
cookies_file="/tmp/wget-cookies.txt"
save_cookies_cmd="--save-cookies $cookies_file --keep-session-cookies"
load_cookies_cmd="--load-cookies $cookies_file --keep-session-cookies"

function mywget {
    log "#!!!!!!!!!# WGET #!!!!!!!!!# wget $quiet $useragent $load_cookies_cmd $@"  
    wget $useragent $load_cookies_cmd "$@"
}

Saddly isn't working. Somehow I'm missing the right way to store parameters on variables $useragent, $save_cookies_cmd, $load_cookies_cmd and caling wget passing these vars as parameters.

I want the result commandline as this:

wget --user-agent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0" --load-cookies /tmp/wget-cookies.txt --keep-session-cookies http://mysite.local/myfile.php

EDIT: My final solution:

In the end my script is correctly working with this:

useragent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0"
useragent_cmd="--user-agent=$useragent"
cookies_file="/tmp/wget-cookies.txt"
save_cookies_cmd="--save-cookies $cookies_file --keep-session-cookies"
load_cookies_cmd="--load-cookies $cookies_file --keep-session-cookies"

function mywget {
    log "#!!!!!!!!!# WGET #!!!!!!!!!# wget $load_cookies_cmd $useragent_cmd $@"  
    wget $load_cookies_cmd "$useragent_cmd" "$@"
}

Thanks for all your responses.

share|improve this question

3 Answers 3

up vote 2 down vote accepted

http://mywiki.wooledge.org/BashFAQ/050

user_agent='--user-agent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0"'
cookies_file="/tmp/wget-cookies.txt"
save_cookies_opts=( --save-cookies "$cookies_file" --keep-session-cookies )
load_cookies_opts=( --load-cookies "$cookies_file" --keep-session-cookies )

function mywget {
    log "#!!!!!!!!!# WGET #!!!!!!!!!# wget $quiet $useragent $load_cookies_cmd $@"
    wget "$user_agent" "${load_cookies_opts[@]}" "$@"
}

Note that in the wget call , "$user_agent" is quoted. This is crucial.

share|improve this answer
    
Clever using the array. Thanks for your response. It certainly will help me in understanding better bash script. –  Filipe Mar 20 at 18:03

The problem lies in the order of expansions (see EXPANSION in man bash for details). Double quotes inside a variable do not work as double quotes when the variable is expanded:

$ a='1 "2 3" 4' ; for b in $a ; do echo $b ; done
1
"2
3"
4

Note that using "$b" does not help you.

Use arrays, so you can let bash do the hard work and you can forget about quoting and escaping:

$ a=(1 '2 3' 4) ; for b in "${a[@]}" ; do echo $b ; done
1
2 3
4
share|improve this answer
    
Thanks for your time. I'm kind of novice in bash script and these tricks will always help me from now on. –  Filipe Mar 20 at 18:05
    mywget() ( echo log "wget $quiet \
                         userstring=${userstring:-unset} \
                         cookies=${cookies:-no} $@"
            echo wget ${userstring:+--useragent="$userstring"} \
                    ${cookies:+--${cookies}-cookies \
                    "$cookies_file" --keep-session-cookies} "$@"
    ) <<-PARAMS
            ${DQ=$(printf \\042)}
            ${NL=
}
            ${userstring="${DQ}Mozilla/5.0 \
                (Windows NT 6.1; WOW64; rv:27.0) \
                Gecko/20100101 Firefox/27.0${DQ}"}
            ${cookies_file="/tmp/wget-cookies.txt"}
            ${cookies=save}
    PARAMS

Please forgive me, but I've taken the liberty of parameterizing your function. Because the values in PARAMS will only set an unset variable, you can override their values very easily - it allows you to set sane defaults and yet still change values per function call as you like. For instance, in the above function, I've set all values to echo and here's the output when run as is:

log wget  userstring="Mozilla/5.0 \
    (Windows NT 6.1; WOW64; rv:27.0) \
    Gecko/20100101 Firefox/27.0" \
    cookies=save
wget --useragent="Mozilla/5.0 \
    (Windows NT 6.1; WOW64; rv:27.0) \
    Gecko/20100101 Firefox/27.0" \
    --save-cookies /tmp/wget-cookies.txt \
    --keep-session-cookies

But if I set "$userstring" to NULL:

    userstring= mywget
log wget  userstring=unset cookies=save
wget --save-cookies /tmp/wget-cookies.txt --keep-session-cookies

Or:

    cookies=load cookies_file=/some/other/cookie.file mywget
log wget  userstring="Mozilla/5.0 \
    (Windows NT 6.1; WOW64; rv:27.0) \
    Gecko/20100101 Firefox/27.0" \
    cookies=load
wget --useragent="Mozilla/5.0 \
    (Windows NT 6.1; WOW64; rv:27.0) \
    Gecko/20100101 Firefox/27.0" \
    --load-cookies /some/other/cookie.file \
    --keep-session-cookies

Or:

    cookies= userstring= mywget
log wget  userstring=unset cookies=no
wget

I've written more on parameter expansion here and here.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.