How to make script with rsync better

Question

I have the following script using rsync for backuping and synchronization of my files.

How can I make this script more compact, faster and with better error handling?

#!/bin/sh
  
errors=""

sudo rsync -avh --delete --no-o --no-g /home/xralf/audio /media/extdevice/rsync_backups/
if [ "$?" -eq "0" ]
then
  echo "no errors in audio"
else
  errors="${errors}error in audio\n"
fi

sudo rsync -avh --delete --no-o --no-g /home/xralf/books /media/extdevice/rsync_backups/
if [ "$?" -eq "0" ]
then
  echo "no errors in books"
else
  errors="${errors}error in books\n"
fi

sudo rsync -avh --delete --no-o --no-g /home/xralf/source_code /media/extdevice/rsync_backups/
if [ "$?" -eq "0" ]
then
  echo "no errors in source_code"
else
  errors="${errors}error in source_code\n"
fi

# more such directories with this code pattern

echo ${errors}

Later I plan not to watch the script execution, so I'd like to see only the result and believe everything went right. I ran the script, but in the end, the only information I have is error in source_code, and I can't see, what exactly the error is (which file caused it).

Answer 1

If your priority is to be compact and fast (ie: minimizing the number of processes, but in this case you are limited by your disk IOs, not by cpu or memory):

#!/bin/bash
sudo rsync -a --delete --no-o --no-g \
       /home/xralf/{audio,books,source_code} \
       /media/extdevice/rsync_backups/ &&
    echo "rsync completed successfully" 1>&2 || echo "rsync ended with errors" 1>&2

Removing the -v option will make the output less noisy, so that you can focus on explicit errors, if any. The 1>&2 redirects STDOUT to STDERR, so that error messages are sent where expected. If running unattended, the following alternatives may be better options than the built-in echo, at the cost of on extra process:

1. a server can use logger to send the message to syslog for centralized handling. Exact syntax will depend of the server configuration.

2. using date to add timing information, with the last line of the script becoming:

date +"[%Y/%m/%d %H:%M:%S] rsync completed successfully" 1>&2 || \
date +"[%Y/%m/%d %H:%M:%S] rsync ended with errors" 1>&2

to output something like [2022/12/05 14:37:27] rsync completed successfully

Answer 2

If you are not forced to use sh but can use a more complex shell like bash, you can do something like this:

#!/bin/bash

dirs=( "audio" "books" "source_code" );
errors=()
for dir in "${dirs[@]}"; do
  sudo rsync \
       -avh --delete --no-o --no-g \
       /home/xralf/"$dir" \
       /media/extdevice/rsync_backups/ &&
    echo "no errors in $dir" ||
      errors+=("error in $dir\n")
done

printf '%s\n' "${errors[@]}"

You probably also want to capture stderr but this does the same thing as your script.

Answer 3

The rsync command accepts the parameter --log-file=FILE. You can examine it for errors when the backup is complete.

For efficiency's sake, it also accepts --exclude-from=FILE and --include-from=FILE. You can create text files containing a list of everything you do and do not want to back up and call them from a single rsync command rather than multiple ones.

See man rsync or the online man page.

Answer 4

Make a file called module.sh and give it the following text:

#!/usr/bin/env bash                                                                                                                     

# -A=Archive. Use recursion and preserve almost everything.
# -V=Verbose.
# -H=Preserve hard links.
# -Z=Compress.
# --Delete=
#     If node in directory 'from' gets deleted,
#     delete it in directory 'to'.
# --No-o=No Owner. 
# --No-g=No Group.
# The --No's are partially undoing the Archive option.

sync_dir(){
    local from="${1}";
    local to="${2}";

    rsync               \
        -avhz           \
        --delete        \
        --no-o          \
        --no-g          \
            "${from}/"  \
            "${to}"     \
        1>/dev/null     \
        ;

    # Store the sync success.
    local sync_result="${?}";

    # The project name is the last directory of the from string.
    local project_name=$(grep -oE '[^/]*$' <<<"${from}");

    local error="No error in: ${project_name}";
    if [ "${sync_result}" -ne "0" ]; then
        error="Error in: ${project_name}\n";
        fi  
    printf "${error}\n";
    };

Then you make another file and source the module from there:

#!/usr/bin/env bash
                                                                       
. ./module.sh;

sync_dir '/home/xralf/audio' '/media/extdevice/rsync_backups/';
# More sync dirs here.

Note different Shebang, this one is POSIX compliant.

I also added Rsync's -z option. It compresses the data en route.