Edit: to give more insight into what I'm trying to achieve, I'll give the example below (there was poor folder structure on part of the team and we aim to fix it):





I have lists (as files, source.txt and destination.txt, and as arrays) for exactly what files need to be copied to which directory, the trouble I'm having is figuring out how to get a source file copied to its respective destination, without having to do it manually because there are hundreds of files that need copying.

The source and destination lists match line by line. For example, line 1 on the source.txt file matches up with line 1 on the destination.txt file.

I've tried the below without success:

rsync -arv --files-from=/home/user/Desktop/source.txt / --files-from=/home/user/Desktop/destination.txt /

^Can only have a single destination, won't work.

for ((i = 0; i < ${#destination[@]}; i++)); do
    rsync -arv --files-from=/home/im-admin/Desktop/source.txt / "${destination[@]}"

^I know I'm doing this wrong, but need help figuring out how to have it copy files directly from line one of the source file to line one of the destination file (or array) because each source and destination folder is unique.

  • 2
    Similarly as cp, rsync accepts only one destination, while you can have many source files or directories. You can use --files-from only for source, which will then be copied to one destination. If you need to have a separate destination for each source, I would recommend you to make a for loop, which would go through source, destination pairs and start a rsync for each pair.
    – nobody
    Aug 3 '20 at 8:47
  • If you're struggling with this, provide us a real-world worked example (with maybe three or four lines from your actual data set) showing what you've got and what you want to achieve.
    – roaima
    Aug 3 '20 at 11:08
  • I find it easier to create a bash file with multiple rsync commands. For /home I like to have another file with a long list of temporary files to exclude. askubuntu.com/questions/545655/…
    – oldfred
    Aug 3 '20 at 18:00
  • 1
    I've added some updates, please let me know if that helps to clarify the question.
    – imdj
    Aug 3 '20 at 19:34
  • What is the relation between source.txt and the list of destinations? Do you also have a text file listing the latter? Do the source and destination lists have an even number of ordered elements, so that the first line of your source list corresponds to the first line of your destination list? Or should source and destination elements be matched based on the variable element of the paths (ShopA, ShopB, ...)?
    – fra-san
    Aug 3 '20 at 19:57

In Bash, if you already have a pair of ordered arrays with a one-to-one correspondence among their elements, using a loop to execute a command for each pair of them is straightforward:

for i in "${!sources[@]}"; do
    echo rsync -av -- "${sources[i]}" "${destinations[i]}"

(Remove the echo to actually copy your files when you are happy with the result you see).

The ! prefix applied to an array variable makes it expand to the list of its indices.

I have omitted the -r option to rsync because it is implied by -a.

Lists of paths stored in text files, one per line (assuming, of course, that your file paths can not contain any newline character), can be read into arrays with:

mapfile -t surces <source.txt
mapfile -t destinations <destination.txt

mapfile (or the equivalent readarray) splits its standard input on newlines (or the delimiter explicitly specified using the -d option). The -t option instructs it to remove the delimiter itself, which would be otherwise preserved as the last character of each array element.

  • Thank you sir, you are a gentleman and a scholar. It works like a charm!
    – imdj
    Aug 3 '20 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.