0

I have just written a simple shell script with Automator on my Mac, which looks like this:

rsync -a --delete "/Volumes/Backup Plus/photo" "/Volumes/My Passport/photo"
rsync -a --delete "/Volumes/Backup Plus/video" "/Volumes/My Passport/video"

It basically makes a backup of some folders from an external drive to another.

At very this time, both drives have exactly the same content, but "rsync"ing takes still a very long time, since it makes a deep recursive comparison, as fas as I understood. (Both photo and video folders are > 1 TB size)

Can I somehow have the option of making this backup only for the folders, whose sizes are changed? This way rsync would have just needed a couple of seconds. I imagine sth like, it first checks the size of parent folder, 74.921.434 bytes. And compares. Same size. And skips directly to the next line in script since no action is needed. And so on..

I know it is not the same thing, I know it is not a backup then.. In worst case, some changes could still end up with the exact same size, but still, it is very very unlikely. And I can still have my original deep rsync backup script, and run it like once in a month or sth.. but the time I need for my daily backup would decrease immensely with this second, faster script. Does this make sense?

1
  • 1
    Most filesystems do not store the size of folders. So finding that information (by using du for example) requires scanning all the files inside anyway. This doesn't avoid the deep scan, just pushes it from rsync to another tool.
    – BowlOfRed
    Feb 2, 2020 at 0:25

1 Answer 1

0

You can check the size with du and then compare them:

#!/bin/sh

pic_source_dir="/Volumes/Backup Plus/photo/"
pic_dest_dir="/Volumes/My Passport/photo/"

vid_source_dir="/Volumes/Backup Plus/video"
vid_dest_dir="/Volumes/My Passport/video"

if [ "$(du "$pic_source_dir")" -ne "$(du "$pic_dest_dir")" ]; then
    rsync -a --delete "$pic_source_dir" "$pic_dest_dir"
fi

if [ "$(du $vid_source_dir)" -ne "$(du "$vid_dest_dir")" ]; then
    rsync -a --delete "$vid_source_dir" "$vid_dest_dir"
fi

If your shell is actually bash (which it is), specifically a bash that can support associative arrays (which the default bash on macos is not), you can clean it up a bit:

#!/usr/bin/env bash

declare -A backup_dirs

backup_dirs=(
    ["/Volumes/Backup Plus/photo/"]="/Volumes/My Passport/photo/"
    ["/Volumes/Backup Plus/video"]="/Volumes/My Passport/video"
)

for src in "${!backup_dirs[@]}"; do
    dst="${backup_dirs["$src"]}"
    if [[ "$(du "$src")" -ne "$(du "$dst")" ]]; then
        rsync -a --delete "$src" "$dst"
    fi
done

Using bash v3.2 from macos you can still sort of clean it up a bit using indirect variable expansion:

#!/bin/bash

pic_src="/Volumes/Backup Plus/photo/"
pic_dst="/Volumes/My Passport/photo/"

vid_src="/Volumes/Backup Plus/video"
vid_dst="/Volumes/My Passport/video"

for i in pic vid; do
    t1="${i}_src"
    t2="${i}_dst"
    src=${!t1}
    dst=${!t2}
    if [[ "$(du "$src")" -ne "$(du "$dst")" ]]; then
        rsync -a --delete "$src" "$dst"
    fi
done

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .