3

I can't undestand well how the parallel command works.

I need to run this simple command: (100 times)

curl https://jsonplaceholder.typicode.com/todos/1
curl https://jsonplaceholder.typicode.com/todos/2
curl https://jsonplaceholder.typicode.com/todos/3
...
curl https://jsonplaceholder.typicode.com/todos/100

end redirect the output to files with the names like these:

1.txt
2.txt
3.txt
....
100.txt
1
  • Unless you are really trying to figure out how to do this with the parallel command, this may solve your problem: for i in {1..100}; do curl "https://jsonplaceholder.typicode.com/todos/$i" > "$i.txt"; done.
    – ozzy
    Commented Jan 3, 2019 at 20:20

2 Answers 2

6

Like this:

parallel curl https://jsonplaceholder.typicode.com/todos/{} ">" {}.txt ::: {1..100}

Consider spending 20 minutes on reading chapter 1+2 of the GNU Parallel 2018 book (print: http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html online: https://doi.org/10.5281/zenodo.1146014). Your command line will love you for it.

1
  • 1
    Your answer triggered me to have a better look at the GNU parallel command; thanks for the link to your book. For the unsuspecting Debian users it is probably worth noting that there are two "parallel" commands: one from the moreutils package (to which my answer relates), and the one known as GNU parallel (to which your answer and book relate). See also this post.
    – ozzy
    Commented Jan 5, 2019 at 8:42
3

Well, this is a somewhat over-engineered Bash-solution, but it works and hopefully clarifies the use of the parallel command:

function xx(){ curl "https://jsonplaceholder.typicode.com/todos/$1" > "$1.txt";}
parallel xx -- {1..100}

The first line creates a new "command" or function called xx which - when executed - causes the execution of a curl command that has its stdout redirected to a file. The xx function takes a single number as its argument; inside the body of the function, it is referred to as `$1', i.e. the first positional parameter.

The second line demonstrates the use of the parallel command, which runs xx once for (and with) each argument from the list 1, 2, 3, ..., 100 (the list 1 2 3 ... 100 is generated by the shell when it performs brace expansion on {1..100}).

NOTE: this answer relates to the parallel command in the moreutils package on Debian systems, not to the GNU parallel command.

1
  • 1
    My version of Tollef's parallel (in moreutils) cannot run bash functions. With GNU Parallel I can make it work: env_parallel xx ::: {1..100} or export -f xx; parallel xx ::: {1..100}.
    – Ole Tange
    Commented Jan 5, 2019 at 10:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .