1

I'd like to rsync a bunch of files and ignore any case, spaces, periods, dashes, or underscore differences in the names when matching what to sync.

So, as an extreme example "TheFilename.zip" would match "__THE- - -File---nam-e...._.zip" (assuming the size and time matched).

I can't think of a way to do this.

  • How about using find to generate a list of files and pipe that you rsync. – user1794469 Jul 7 '17 at 3:14
  • 1
    You can't do that with rsync. rsync always assumes source filenames match destination filenames. The easy workaround would be to normalize filenames on both sides before running rsync. The hard workaround would be to essentially re-implement rsync in shell: make a list of files on destination with sizes, timestamps, and checksums, make the same list on source, compare them according to your rules, decide what to sync, then feed the list to rsync (or just copy the files by some other means, since it no longer matters at that point). – Satō Katsura Jul 7 '17 at 9:23
  • I figured it was nigh-impossible. My idea, but it's absurd, is to create MD5 hashes, then manually sync the files to a neutral location, then copy them to the existing filename. I'm not going to do it though :) – lbutlr Jul 7 '17 at 23:05
1

This script will probably do what you want

#!/bin/bash
#
ritem="$1"              # [ user@ ] remotehost : [ / ] remotepath_to_directory
shift

rhost="${ritem/:*}"     # Can be remotehost or user@remotehost
rpath="${ritem/*:}"     # Can be absolute or relative path to a directory

# Get list of files on remote
#
echo "Looking in $rpath on $rhost" >&2
ssh -n "$rhost" find "$rpath" -maxdepth 1 -type f -print0 |
    while IFS= read -r -d $'\0' rfile
    do
        rkey=$(printf "%s" "$rfile" | tr -d '[:space:]_. -' | tr '[:upper:]' '[:lower:]')
        list[${rkey/*\/}]="$rfile"
    done


# Get list of files from local and copy to remote
#
ss=0

echo "Considering $*" >&2
for lpath in "$@"
do
    test -f "$lpath" || continue

    lfile="${lpath/*\/}"
    lkey=$(printf "%s" "$lfile" | tr -d '[:space:]_. -' | tr '[:upper:]' '[:lower:]')

    # Do we have a match in the map
    rfile="${list[$lkey]}"
    test -z "$rfile" && rfile="$lfile"

    # Copy across to the remote system
    echo "Copying $lpath to $rhost:$rpath/$rfile" >&2
    rsync --dry-run -av "$lpath" "$rhost":"$rpath/$rfile" || ss=$((ss+1))
done


# All done. Exit with the number of failed copies
#
exit $ss

Example usage

375871.sh remotehost:remotepath localpath/*

Remove --dry-run when you're happy it's working as expected.

0

As Satō Katsura points out in comments, this is impossible to do with rsync out of the box. It is also something that I think rsync should not do.

If you are copying a single file, and if that filename is available in the variable name as __THE- - -File---nam-e...._.zip, then you may remove the unwanted characters from the filename and copy the file like this:

ext=${name##*.}
shortname=${name%.$ext}

rsync -a "$name" "user@target:${shortname//[-_ .]}.$ext"

For name='__THE- - -File---nam-e...._.zip', $ext will be zip and $shortname will be __THE- - -File---nam-e...._.

If your shell does not support ${parameter//word}, then use

rsync -a "$name" "user@target:$(printf '%s' "$shortname" | tr -d '-_ .' ).$ext"

Both ${shortname//[-_ .]}.$ext and $(printf '%s' "$shortname" | tr -d '-_ .' ).$ext will become THEFilename.zip.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.