2

I want to using crontab to synchronize two directory between my linux partion and windows partion like this:

24 9 * * * cp -r /home/fan/Data /media/T/Data

But it would create a directory named Data in the origin Data directory, instead of copy the missing file from the source directory. I can't find a proper option at the cp manual to perfectly solve this. How can i copy the missing file(they do exist at the destination directory) from the src to dir.

And by the way, seems it need the T disk have been mounted to run the copy command, how to automatically mount the disk when i need to run the command(the mount command should run as root).

And how can i get the error message if the command have an exception?

1

To address the error-message portion of the question, you might choose to run a script from cron instead of the system command.

24 9 * * * /usr/local/sbin/sync_data.sh

Create the file as /usr/local/sbin/sync_data.sh, giving root ownership and execute permission: chown root:root /usr/local/sbin/sync_data.sh && chmod 0700 /usr/local/sbin/sync_data.sh. The contents of the script are below.

#!/usr/bin/env bash
if [[ $EUID -ne 0 ]]; then
    echo "This script must be run as root." 1>&2
    exit 1
fi

# [ ] vs [[ ]] : http://mywiki.wooledge.org/BashFAQ/031

dir_src="/home/fan/Data/"
dir_dst="/media/T/"

# A Boolean to know if running interactively or not
b_interactive=1
if [[ -v PS1 ]] ; then # The "-v PS1" method is for BASH 4.2+
    $b_interactive=0   # Could also use [[ -z "$PS1" ]]
fi

# Send messages to console or syslog.
function notify() {
    message="$1"
    if [[ $b_interactive -eq 1 ]] ; then
        echo "$message"
    else
        # eval combines args into a single string for execution...
        eval $cmd_logger -p err "$0: $message"
    fi
}

# If the mount point if not currently mounted...
if [[ "$(grep $dir_dst /proc/mounts)" = "" ]] ; then
    # Try to mount the directory.
    mount $dir_dst

    # Send a message to console or syslog.
    if [[ $? -ne 0 ]] ; then
        notify("$0 failed to mount $dir_dst")
        exit 1;
    fi
fi

# Create a backup directory if it does not exist
if [[ -d $dir_dst ]] ; then
    mkdir -p $dir_dst 2>/dev/null
fi

# A one-way sync to dir_dst, deleting files that no longer exist in DIR_SRC...
rsync -a --delete $dir_src $dir_dst &> /dev/null

# Check the return status of last command...
if [ $? -eq 0 ]; then
    notify("$0: the rsync process succeeded.");
else
    notify("$0: the rsync process failed.");
fi

unset dir_src
unset dir_dst
unset b_interactive
  • Thanks. But i think the variable assignment should not with a "$"(like DIR_SRC="/home/fan/Data/" not $DIR_SRC="/home/fan/Data/" ), and the test condition should be [[ "$(grep $DIR_DST /proc/mounts)" = "" ]], or it would report errors in my shell. But i can't check the status about mount. This $(mount $DIR_DST) would be "" no matter it is success or fail. That really confused me. – Ziu Apr 7 '14 at 8:18
7

The go to tool for syncing data is rsync. You can sync either at the directory level or just the contents of a directory like so:

Examples

directory sync
24 9 * * * rsync -a /home/fan/Data /media/T/
contents sync
24 9 * * * rsync -a /home/fan/Data/ /media/T/Data/

The first example will sync the directory Data from /home/fan to the directory /media/T. The second example will sync the contents of Data to the directory /media/T/Data.

As an additional tip I'll sync to target directories using this notation:

24 9 * * * rsync -a /home/fan/Data/ /media/T/Data/.

The /. I find more obvious when reading through my rsync commands.

Behavior Examples

Using rsync is probably one of the most confused issues I deal with when training new users.

Sample data:

$ tree home/fan/Data/
home/fan/Data/
|-- file1
|-- file2
|-- file3
|-- file4
`-- file5
1. syncing contents of dir Data
$ rsync -a home/fan/Data/ media/T/

Notice we put the contents of Data into T.

$ tree media/T/
media/T/
|-- file1
|-- file2
|-- file3
|-- file4
`-- file5

0 directories, 5 files
2. syncing the directory
$ rsync -a home/fan/Data media/T/

Notice we put the directory Data into T.

$ tree media/T/
media/T/
`-- Data
    |-- file1
    |-- file2
    |-- file3
    |-- file4
    `-- file5

1 directory, 5 files
  • @TAFKA'goldilocks' - thanks edited that in. – slm Apr 6 '14 at 15:20
  • @TAFKA'goldilocks' -a implies -p which preserves all permissions including executability. What you can add is -HAX, which will preserve hard links, access control lists and extended attributes respectively. – Graeme Apr 6 '14 at 15:23
  • @TAFKA'goldilocks' - graeme's correct, the man page says the -a includes --perms which would cover the execute bit. That's kind of what I thought, but had to check. – slm Apr 6 '14 at 15:27
  • Yeah you're right, did a quick test. Dunno why I've been using -aE all this time. But hey, look: macinstruct.com/node/85 In fact a search for rsync tutorial "-aE" reveals this misinformation all over the place. – goldilocks Apr 6 '14 at 15:33
  • The -H one is can actually be a good idea on a root filesystem. I recently discovered that the bzip2, bunzip and bzcat binaries are all hard linked on my system - using -H would preserve things like this. – Graeme Apr 6 '14 at 15:35
2

Using rsync:

24 9 * * * rsync -a /home/fan/Data/ /media/T/Data

Add the extra / at the end of the name of the source directory will tell rsync to copy the contents rather than the full directory. rsync will also avoid copying files that have the same timestamps/size (ie ones that don't need to be copied in 99.99% of cases).

Also consider using the --inplace option for rsync to speed things up if you are syncing files on the same system. From the man page:

--inplace
    This option changes how rsync transfers a file when its data needs to be
    updated: instead of the default method of creating a new copy of the file and
    moving it into place when it is complete, rsync instead writes the updated
    data directly to the destination file.

There are pros and cons to this option though, probably the biggest con is that if the sync is interrupted, the target file will be left in an inconsistent state which can only be fixed by starting over again.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.