2

Let's say I have a file listing the path of multiple files like the following:

/home/user/file1.txt
/home/user/file2.txt
/home/user/file3.txt
/home/user/file4.txt
/home/user/file5.txt
/home/user/file6.txt
/home/user/file7.txt

Let's also say that I want to copy those files in parallel 3 by 3. I know that with the command parallel I can execute a specific command in parallel as the following:

parallel bash -c "echo hello world" -- 1 2 3

However, this way of running parallel is hardcoded because even if I use a variable inside the quotes, it will only have a fixed parameter. I'd like to execute the parallel command getting parameters dynamically from a file. As an example, let's say I'd like to copy all files from my file running three parallel processes (something like cp "$file" /home/user/samplefolder/). How can I do it? Is there any parameter I can use with parallel to accomplish that and get parameters dynamically from a file?

1 Answer 1

2

If you use GNU Parallel you can do one of these:

parallel cp {} destination/folder/ :::: filelist
parallel -a filelist cp {} destination/folder/
cat filelist | parallel cp {} destination/folder/

Consider spending 20 minutes on reading chapter 1+2 of the GNU Parallel 2018 book (print: http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html online: https://doi.org/10.5281/zenodo.1146014). Your command line will love you for it.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .