I want to use parallel in this example:
cat codigos.txt | parallel -j 10 "wget http://mywebsite.com/teste?id={}&x=&date=01/01/2017" --load-cookies=cookies.txt
I have three files: codigos.txt
, a file with the ids, dates.txt
, a file with the dates and cookies.txt
the file with the cookie that I need to do the request.
What I want to do is to save the response from mywebsite.com for each url, using the contents of my files.
Problems:
- How do I do to use information coming from two files?
- The url is cut at the '&' (even when I put the quotes). Example:
request made to http://mywebsite.com/teste?id=1
As this example (but without put the space or cut because of the '&'):
cat abc-file | parallel -a - -a def-file echo
reference: https://www.gnu.org/software/parallel/parallel_tutorial.html
'strong quotes'
, which prevents the&
s from being parsed by the shell.mywget() { wget "http://mywebsite.com/teste?id=$1&x=&date=$2" --load-cookies=cookies.txt; }; export -f mywget; parallel mywget :::: codigos.txt dates.txt