I'd like to move new music from my seedbox to my local server on an hourly cron job. My local music is moved into new folders by a musicbrainz script as soon as it finishes syncing on my local machine, meaning the next sync job needs to only sync new files from the seedbox, ignoring removed files on the local server. Both machines are Linux.

My current solution is to use the rsync --exclude-from flag to read a sync log file, appended to by the previous rsync. It's mostly successful however it appears some characters are causing some directories & files to not match causing re-syncing. It appears as though it may be a UTF-8 issue, so I've tried adding: env LC_ALL=en_US.UTF-8 rsync... and various --iconv configurations however I still can't get a reliable sync, here's my script:

echo "Rsync My Artists"
rsync -vrz \
--exclude-from=/mnt/user/sync/music-myartists/__sync-log \
-e ssh \
[email protected]:/home37/xxxx/downloads/deluge-filters/my-artists/ /mnt/user/sync/music-myartists/ \
| grep -v ^sending \
| grep -v ^sent \
| grep -v ^total \
| grep -v 'receiving incremental file list' \
>> /mnt/user/sync/music-myartists/__sync-log
echo "Beets Moving My Artists"
docker exec beets beet -c /config/config-move-quiet.yaml import /sync/music-myartists
find /mnt/user/sync/music-myartists/ -not -name '__sync-log' -delete

Here's an example of a __sync-log file getting rewritten with the same content: https://pastebin.com/FMSfwUU8

Any ideas?

  • I can't see the problem you're describing in that large log file on your pastebin. Perhaps you could cut out a relevant segment of your log file and put it here in your question so that it's easy to see. May 15, 2019 at 12:09
  • for example: Larry Heard ?? Where Life Begins (2003)[FLAC]{ML-9014}/03 Larry Heard - Insight.flac is actually Larry Heard — Where Life Begins (2003)[FLAC]{ML-9014}/03 Larry Heard - Insight.flac May 15, 2019 at 22:43

1 Answer 1


Ok I worked it out, it had nothing to do with utf-8, rather filenames triggering one of rsyncs three wildcard characters '*', '?', and '[' . The '?' and '*' weren't causing issues, however the '[' introduces a character class, such as [a-z] or [[:alpha:]], which caused mismatches.

After writing rsyncs progress to the log i added:

sed -i 's/\[/\*/g' /mnt/user/sync/music-manual/__sync-log

To replace '[' with '*'

is there a neat way of including this in the rsync command?


  • You can probably escape the [ to \[ with sed 's/\[/\\[/g', but you are right, I don't think there is any way to get rsync to interpret the patterns as literal strings.
    – meuh
    May 16, 2019 at 16:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .