I'd like to move new music from my seedbox to my local server on an hourly cron job. My local music is moved into new folders by a musicbrainz script as soon as it finishes syncing on my local machine, meaning the next sync job needs to only sync new files from the seedbox, ignoring removed files on the local server. Both machines are Linux.
My current solution is to use the rsync --exclude-from flag to read a sync log file, appended to by the previous rsync. It's mostly successful however it appears some characters are causing some directories & files to not match causing re-syncing. It appears as though it may be a UTF-8 issue, so I've tried adding: env LC_ALL=en_US.UTF-8 rsync... and various --iconv configurations however I still can't get a reliable sync, here's my script:
echo "Rsync My Artists"
#
rsync -vrz \
--exclude-from=/mnt/user/sync/music-myartists/__sync-log \
-e ssh \
xxxx@xxxx.xxx.xxx:/home37/xxxx/downloads/deluge-filters/my-artists/ /mnt/user/sync/music-myartists/ \
| grep -v ^sending \
| grep -v ^sent \
| grep -v ^total \
| grep -v 'receiving incremental file list' \
>> /mnt/user/sync/music-myartists/__sync-log
#
echo "Beets Moving My Artists"
#
docker exec beets beet -c /config/config-move-quiet.yaml import /sync/music-myartists
#
find /mnt/user/sync/music-myartists/ -not -name '__sync-log' -delete
################################
Here's an example of a __sync-log file getting rewritten with the same content: https://pastebin.com/FMSfwUU8
Any ideas?