1

Let's say I have a file listing the path of multiple files like the following:

/home/user/file1.txt
/home/user/file2.txt
/home/user/file3.txt
/home/user/file4.txt
/home/user/file5.txt
/home/user/file6.txt
/home/user/file7.txt

Let's also say that I want to copy those files in parallel 3 by 3. I know that with the command parallel I can execute a specific command in parallel as the following:

parallel bash -c "echo hello world" -- 1 2 3

However, this way of running parallel is hardcoded because even if I use a variable inside the quotes, it will only have a fixed parameter. I'd like to execute the parallel command getting parameters dynamically from a file. As an example, let's say I'd like to copy all files from my file running three parallel processes (something like cp "$file" /home/user/samplefolder/). How can I do it? Is there any parameter I can use with parallel to accomplish that and get parameters dynamically from a file?

1 Answer 1

1

If you use GNU Parallel you can do one of these:

parallel cp {} destination/folder/ :::: filelist
parallel -a filelist cp {} destination/folder/
cat filelist | parallel cp {} destination/folder/

Consider spending 20 minutes on reading chapter 1+2 of the GNU Parallel 2018 book (print: http://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html online: https://doi.org/10.5281/zenodo.1146014). Your command line will love you for it.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.