Rsync only new files from a remote server

Question

I have a remote server with an rsync share and I need to pull just the most recent 7 days of files.

I can SEND files based on date by using:

find /path -mtime 7 ... -exec rsync -a

but I can't just login to the server and send the files to my local machine so I need a way to use rsync to get all the files that are less than 7 days old and grab them.

I cannot simply sync everything as the total data on the server exceeds 100TB and I don't have access to the remote server other than the rsync module.

Answer 1

Surprisingly, this can actually be done with rsync(1) alone, without shell access to the remote server.

Assuming bash(1) and GNU date(1):

#! /usr/bin/env bash

src=rsync://example.com/archives/
dst=/path/to/mirrors

cutoff=$( date -d '7 days ago' +%s )
rsync -na --no-motd --out-format='%M %f' "$src" "$dst" | \
    while IFS= read -r line; do
        d=${line%% *}
        fn=${line#* }
        fdate=$( date -d "${d/-/ }" +%s ) || continue
        if [ $fdate -ge $cutoff ]; then printf '%s\0' "$fn"; fi
    done | \
    rsync -a --files-from=- -0 "$src" "$dst"

Answer 2

This is the approach that I used as part of an rsnapshot backup script:

rhost=user@backuphost
rpath=/path/to/files

ssh -qx "$rhost" "cd '$rpath' && find . -mtime -7 -print0" |
    rsync --from0 --files-from=- -avHS "$rhost:$rpath/" .