Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am trying to copy two files using rsync from a machineB but somehow it always copies only one file but not the second file -

Here is the syntax I am using -

rsync -avz david@machineB:'/data/pe_t1_snapshot/20140317/t1_weekly_1680_0_200003_5.data :/data/pe_t1_snapshot/20140317/t1_weekly_1680_1_200003_5.data' /data01/primary

It only copies this file t1_weekly_1680_0_200003_5.data into /data01/primary folder and it didn't copied second file which is t1_weekly_1680_1_200003_5.data.

I am using Ubuntu 12.04 and rsync version is -

rsync  version 3.0.9  protocol version 30

This is the error I am getting -

receiving incremental file list
rsync: change_dir "/home/david/:/data/pe_t1_snapshot/20140317" failed: No such file or directory (2)
t1_weekly_1680_0_200003_5.data

sent 30 bytes  received 504982813 bytes  6196108.50 bytes/sec
total size is 1761988281  speedup is 3.49
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1536) [generator=3.0.9]

I started reading the rsync manual but somehow I am not able to understand what wrong I am doing?

Update:-

It works fine on a single command if I run it from the console but I need to run that from the shell script so I added your suggestion in my shell script.

I just ran your suggestion on my below example and I got an error.

Below is the shell script which simply tries to copy file from machineB, if the files are not there in machineB then it should be there in machineC so it will try to copy from machineC. PRIMARY_PARTITION and SECONDARY_PARTITION will have file numbers.

And below shell script I am running on machineA.

#!/usr/bin/env bash

readonly PRIMARY=/data01/primary
readonly FILERS_LOCATION=(machineB machineC)
readonly MEMORY_MAPPED_LOCATION=/bexbat/data/be_t1_snapshot
PRIMARY_PARTITION=(0 548)
SECONDARY_PARTITION=(1101 1374)

dir1=$(ssh -o "StrictHostKeyChecking no" david@${FILERS_LOCATION[0]} \
  ls -dt1 "$MEMORY_MAPPED_LOCATION"/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | 
  head -n1)
dir2=$(ssh -o "StrictHostKeyChecking no" david@${FILERS_LOCATION[1]} \
  ls -dt1 "$MEMORY_MAPPED_LOCATION"/[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] |
  head -n1)

echo $dir1
echo $dir2

## Build your list of filenames before the loop. 
for n in "${PRIMARY_PARTITION[@]}"
do
    primary_files="$primary_files :$dir1"/t1_weekly_1680_"$n"_200003_5.data
done

## Repeat for $SECONDARY_PARTITION
for n in "${SECONDARY_PARTITION[@]}"
do
    secondary_files="$secondary_files :$dir2"/t1_weekly_1680_"$n"_200003_5.data
done

echo "Primary: " $primary_files
echo "Secondary: " $secondary_files


if [ "$dir1" = "$dir2" ]
then
    find "$PRIMARY" -mindepth 1 -delete

    # this line is giving an exception somehow
    rsync -avz david@${FILERS_LOCATION[0]}${primary_files} $PRIMARY/
    rsync -avz david@${FILERS_LOCATION[1]}${primary_files} $PRIMARY/

fi

Below is the error I got -

Unexpected remote arg: :/data/pe_t1_snapshot/20140320/t1_weekly_1680_0_200003_5.data
rsync error: syntax or usage error (code 1) at main.c(1232) [sender=3.0.9]
Unexpected remote arg: :/data/pe_t1_snapshot/20140320/t1_weekly_1680_0_200003_5.data
rsync error: syntax or usage error (code 1) at main.c(1232) [sender=3.0.9]
Unexpected remote arg: :/data/pe_t1_snapshot/20140320/t1_weekly_1680_1101_200003_5.data
rsync error: syntax or usage error (code 1) at main.c(1232) [sender=3.0.9]
Unexpected remote arg: :/data/pe_t1_snapshot/20140320/t1_weekly_1680_1101_200003_5.data
rsync error: syntax or usage error (code 1) at main.c(1232) [sender=3.0.9]
share|improve this question
    
Also, because there's only a one character difference: --include="fi"[10]"le" –  mikeserv Mar 21 at 20:38

1 Answer 1

up vote 4 down vote accepted

By the looks of things, all you need to do is drop the quotes (line breaks added for clarity):

 rsync -avz \
   david@machineB:/data/pe_t1_snapshot/20140317/t1_weekly_1680_0_200003_5.data \
   :/data/pe_t1_snapshot/20140317/t1_weekly_1680_1_200003_5.data \
   /data01/primary

From the rsync man page:

The  syntax  for requesting multiple files from a remote host is done by
specifying additional remote-host args in the same style as the first, or with
the hostname omitted.  For instance, all these work:

          rsync -av host:file1 :file2 host:file{3,4} /dest/

The first two files in the copied example use the same syntax as you have, however they are separate arguments (quoting them concatenates them into a single argument). If your paths contain characters which need to be quoted you can do something like:

rsync -avz \
  'user@host:dodgy path/file_with_asterix*' \
  ':some_other/dodgy\\path' \
  /dest

Update

I think the simplest way to make your script work is just to use arrays for primary_files and secondary_files. The relevant changes for primary_files are:

for n in "${PRIMARY_PARTITION[@]}"
do
  primary_files+=( ":$dir1/t1_weekly_1680_${n}_200003_5.data" )
done

....

echo "Primary: ${primary_files[@]}"

...

rsync -avz "david@${FILERS_LOCATION[0]}${primary_files[@]}" "$PRIMARY/"

The [@] will split the array into different arguments regardless of quoting. Otherwise, mind your variable quoting, some of what you have may or may not cause issues.

share|improve this answer
    
Thanks for your suggestion. Yeah it worked fine with your suggestion if I run it directly on the console with that single command but I need to run that from my shell script and if I use your suggestion, it is giving an error if I run it from shell script. I have updated my question with the shell script and what I am doing. –  Webby Mar 21 at 22:57
    
Any thoughts what is going on? –  Webby Mar 21 at 23:38
    
@Webby, updated –  Graeme Mar 21 at 23:41
    
I just tried your suggestion and when this line got printed out echo "Primary: ${primary_files[@]}" it is missing the file number and other number after 1680. For example this is what it is getting printed out - /data/pe_t1_snapshot/20140320/t1_weekly_1680_.data. It is missing lot of numbers after 1680. –  Webby Mar 21 at 23:48
    
@Webby, ok you want ${n} instead of $n. Updated. –  Graeme Mar 21 at 23:54

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.