first post sorry if I did some mess here =)
Using Ubuntu 14.04lts 64bits server version.
I have a list (url.list) with only URLs to download, one per line, that looks like this:
http://domain.com/teste.php?a=2&b=3&name=1 http://domain.com/teste.php?a=2&b=3&name=2 ... http://domain.com/teste.php?a=2&b=3&name=30000
As you can see, there are many lines in the file (in this case 30000). Because of that I'm using a trick to download many URLs simultaneosly with this:
cat url.list | xargs -n 1 -P 10 <<MAGIC COMMAND THAT WILL SAVE ME>>
The problem is that I'd like to rename the output file with the same value of the name field, like: 1.html, 2.html, ..., 30000.html ecc, and use curl to limit the size of the file to 50KB. So the curl command should be something like:
curl -r 0-50000 -L $URL -o $filename.html -a $filename.log
How can I have it done?
I can parse the output of the pipe with echo $URL | sed -n -e 's/^.*name=//p' but I don't know how use this in the same line grabbing the output of a pipe in 2 variables ($URL and $filename).
I tried this with no success:
cat url.list | xargs -n 1 -P 10 | filename=$(sed -n -e 's/^.*name=//p') ; curl -r 0-50000 -L $URL -o $filename.html -a $filename.log
Thank you in advance