1

I'm trying to create a shell script to run via my file manager's context menu to make certain replacements in some file names that might cause problems when transferred across different platforms (Linux, MacOS, Windows etc.).

I have so far managed the obtain the following (probably quite dirty) code with my beginner-level scripting skills, but I'm still struggling to find a way to replace these characters: ‘ ’ “ ” / (When I use these smart single and double quotes, and slash like other sed expressions, the file is deleted!).

And another problem I'm struggling is that this script fails to remove a space before the file name although it works when entered directly in terminal.

I would appreciate some help in finding a solution for this.

for filename in "${@}"; do
    NEWNAME="$(echo "$filename" | sed -e 's/:/-/g' -e "s/'//g" \
    -e 's/["|?|*]//g' -e 's/[<|>]/ /g' -e 's/\\/ /g' -e 's/\[/ /g' \
    -e 's/\]/ /g' -e 's/\s\s*/ /g' -e 's/^\s\s*//g' -e 's/\s\s*\./\./g')"
    mv "$filename" "$NEWNAME"
done
3
  • 1
    Relating unix.stackexchange.com/a/150650/117549
    – Jeff Schaller
    Oct 22 '20 at 16:06
  • 1
    First tip in to use Larry Wall's rename command. It is sed for filenames. (be careful, there are many programs with this name). Oct 22 '20 at 16:38
  • 1
    I did what I could from what I could guess of your intentions. I could probably give you a better solution if you explained exactly what things you want to remove/replace. I can only guess from your sed.
    – terdon
    Oct 22 '20 at 17:43
2

Just use perl-rename (found as rename on Debian and Ubuntu etc). First, for testing, let's create an awful name:

touch "   a truly “horrible”, ‘awful’"$'\n'"name     with a newline and *globs*, and even a 'single' quote or two!   .txt"

This is what that looks like:

$ ls
'   a truly “horrible”, ‘awful’'$'\n''name     with a newline and *globs*, and even a '\''single'\'' quote or two!   .txt'

Note that that there is a literal newline character, if you try to loop over this (badly), you will see:

$ for f in *; do echo "$f"; done
   a truly “horrible”, ‘awful’
name     with a newline and *globs*, and even a 'single' quote or two!   .txt

So, that name has most if not all the problems you'll be facing. Now, use rename to get rid of the bad characters:

$ rename 's/[*“”‘’\n<|>"[\]]//g; s/:/-/g; s/\s+/ /g; s/^\s*//; s/\s+\././g; '"s/'//g" *
$ ls -N
a truly horrible, awfulname with a newline and globs, and even a single quote or two!.txt

As you can see, that removed all of the bad things you were looking for (as far as I could tell since I only had your sed attempt to go on). You could fit this into your script like this:

for filename in "${@}"; do
    rename 's/[*“”‘’\n<|>"[\]]//g; 
    s/:/-/g; 
    s/\s+/ /g; 
    s/^\s*//; 
    s/\s+\././g; '"s/'//g" "$filename"
done

Explanation

The basic syntax is very similar to sed, you are using the same substitution operator. The regular expressions are:

  • s/[*“”‘’\n<|>"[\]]//g; : substitute every occurrence of *, , , , , \n, <, |, >, ", [, or `] with nothing, delete them.
  • s/:/-/g: substitute every occurrence of any whitespace character (basically a space, a tab or a newline) with -.
  • s/\s+/ /g: substitute all occurrences of one or more consecutive whitespace characters with a single space. *s/^\s*//: remove all leading whitespace from the beginning of the file name,
  • s/\s+\././g : remove all occurrences of one or more whitespace characters that come before a ..
  • "s/'//g": remove all single quotes. Note how the whole command is rename '...' and then I have added "s/'//g". This is because you cannot escape a single quote within a single-quoted string, so I had to close the single-quoted string and open a new, double quoted one to deal with the ' characters.

Also, I did not bother to deal with /, since / along with \0 are the only characters that are not allowed in filenames and you simply cannot create a file name that contains a /.

4
  • Thank you so much! However, although I wasn't allowed to do it in terminal, Dolphin file manager allowed me to create a file with the name "invalid/character/in/filename.txt", and I could even move it to another directory! And when I ran your script in that directory, each slash was replaced with a character shown as a square in Dolphin, and as a question mark in terminal. Therefore, it might also a good idea to include it in this script if it's possible at all. I will also work on this interesting puzzle further. ;-)
    – Sadi
    Oct 23 '20 at 18:30
  • @Sadi I'm afraid that isn't possible. You simply cannot create a file that contains a slash in its name. Are you sure that Dolphin didn't create the directory invalid/character/in/ and the file filename.txt in that directory? Alternatively, maybe it used something like (fileformat.info/info/unicode/char/2215/index.htm)? But it couldn't have been a /.
    – terdon
    Oct 23 '20 at 18:57
  • Oh, sorry; I've just discovered that Dolphin actually replaces slash character (/) with fraction slash character ( ⁄ ) automatically. It seems this character cannot be included directly, and needs some special method, if at all.
    – Sadi
    Oct 23 '20 at 20:38
  • OK, after a little research, I've found that to get rid of these as well I need to add s/\xe2\x81\x84/ /g; as the first expression.
    – Sadi
    Oct 23 '20 at 21:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.