I have been asked by the data owner to copy a specific folder (and its large amount of subfolders and files) via FTPS to our cloud storage provider. I am using LFTP for that, and the upload worked well until I hit a snag.

There are several folders with multiple files that have the same filename except for case. For example, folder data has the following files: testfile1.txt, TestFile1.TXT

When I try to upload those via LFTP, I get an error that the file already exists. So for my purposes, I need the files to be case-insensitive before uploading. To address this issue, I would like to use a script that searches the current directory recursively, and moves any case-insensitive duplicates to a subfolder. In my example above, I would want the script to create a subfolder called Duplicates and then move TestFile1.TXT into it. I suppose it's possible that I could have multiple duplicated filenames, so the script should create a Duplicates2 folder for the second duplicated filename, and so on.

Also, I should note that for the few "duplicated" files that I checked, they had differing filesizes. I am not going to make any assumptions about the files being actual duplicates, which is why I want to move them rather than delete them.

  • 1
    What is the filesystem you're copying to? Most native POSIX filesystems are entirely case-sensitive, so testfile1.txt and TestFile1.txt are in fact two different, nonconflicting filenames.
    – DopeGhoti
    Commented Apr 9, 2018 at 15:35
  • 1
    I just tested the cloud storage platform's case-sensitivity by uploading a file, changing the filename case, then uploading the original file again. It overwrote the renamed file.
    – Joe
    Commented Apr 9, 2018 at 17:58

2 Answers 2


The bash script below loops through the files in the current directory, looking for duplicate filenames case insensitively. If a match is found, it looks to create a "Duplicates" folder that doesn't exist already, then moves the duplicate file into that directory.

The outer loop is there in order to re-compute the file globs (*) for the loops, in case a file gets moved. The outer loop runs until no files are moved.


while [ $changes -gt 0 ]
  for one in *
    for two in *
      shopt -u nocasematch
      # if it's the exact same filename, skip
      [[ "$one" == "$two" ]] && continue
      shopt -s nocasematch
      # if the file name matches case-insensitively, then mv it
      if [[ "$one" == "$two" ]]
        while [ -d Duplicates"${suffix}" ]
          suffix=$((suffix + 1))
        mkdir Duplicates"${suffix}"
        mv "$two" Duplicates"${suffix}"

With these sample files:


A sample run of the script creates:

$ tree .
├── afile.txt
├── Duplicates
│   └── TestFile1.TXT
├── Duplicates1
│   └── testfile1.txt
└── TESTfile1.txT
  • Thank you for the script! Unfortunately I'm getting a syntax error, and being a Linux noob, I don't know how to resolve it.
    – Joe
    Commented Apr 11, 2018 at 15:07
  • Did you run it with bash, @Joe? What's the error you get?
    – Jeff Schaller
    Commented Apr 11, 2018 at 15:12
  • I created a .sh file with the code, and here's what I got: [username@text test]$ MvDuplicates.sh /home/username/scripts/MvDuplicates.sh: line 6: syntax error near unexpected toke' do 'home/username/scripts/MvDuplicates.sh: line 6: do
    – Joe
    Commented Apr 11, 2018 at 15:16
  • I then explicitly ran it with bash to be sure: "bash -x MvDuplicates.sh"
    – Joe
    Commented Apr 11, 2018 at 15:17
  • [username@text test]$ bash -x MvDuplicates.sh + LMOD_PKG=/usr/share/lmod/lmod + LMOD_DIR=/usr/share/lmod/lmod/libexec + LMOD_CMD=/usr/share/lmod/lmod/libexec/lmod + MODULESHOME=/usr/share/lmod/lmod + export LMOD_PKG + export LMOD_CMD + export LMOD_DIR + export MODULESHOME + '[' -z -XqeMRF ']' + '[' -z more ']' + LMOD_VERSION=6.0.15 + export LMOD_VERSION + '[' : '!=' : ']' + unalias ml + true ++ echo YES ++ tr '[:upper:]' '[:lower:]' + export_module=yes + '[' -n '4.1.2(1)-release' -a yes '!=' no ']' + export -f module + export -f ml
    – Joe
    Commented Apr 11, 2018 at 15:17

Rather than creating a very complex script to sort out case sensitive you can actually make the FTP case sensitive so you don't need to do this. Refer the LFTP man page for specifics on options: Check if your LFTP command actually is has no case flag in it.

-i, --nocase case-insensitive pattern matching

You can transfer the folders as is file names should valid on the host. Adding more details about destination host would help.

Also I'd recommend to check if you can implement this solution base on file extension.

LFTP mget ignoring case


I'm sure you don't have that issue as well.

If still the actual requirement is to have a script. You can create a python script using set.

  • 1
    Thank you, but this won't help since my FTP destination filesystem is not case sensitive.
    – Joe
    Commented Apr 11, 2018 at 15:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .