2

I have a deep folder structure on a Debian machine where the directory names and the filenames contain some "special" characters (ä,ö,ü).  However, these are not in "ISO-8859-1" but in a Unicode-combined-character form.  So as I learned, that's a simple a along with the two dots (diaeresis/umlaut) as "separate" characters.

I tried to bulk rename all files and folders by using find and sed:

#!/bin/bash

# Files - normal characters
find . -depth -name "*[äöüÄÖÜ]*" -exec bash -c 'mv "$1" "$(echo $1 | sed -e "s/ä/ae/g; s/ö/oe/g; s/ü/ue/g; s/Ä/Ae/g; s/Ö/Oe/g; s/Ü/Ue/g")"' _ {} \;
# Files - Unicode combining characters
find . -depth -name "*[äöüÄÖÜ]*" -exec bash -c 'mv "$1" "$(echo $1 | sed -e "s/ä/ae/g; s/ö/oe/g; s/ü/ue/g; s/Ä/Ae/g; s/Ö/Oe/g; s/Ü/Ue/g")"' _ {} \;

# Directories - normal characters
find . -depth -type d -name "*[äöüÄÖÜ]*" -exec bash -c 'mv "$1" "$(echo $1 | sed -e "s/ä/ae/g; s/ö/oe/g; s/ü/ue/g; s/Ä/Ae/g; s/Ö/Oe/g; s/Ü/Ue/g")"' _ {} \;
# Directories - Unicode combining characters
find . -depth -type d -name "*[äöüÄÖÜ]*" -exec bash -c 'mv "$1" "$(echo $1 | sed -e "s/ä/ae/g; s/ö/oe/g; s/ü/ue/g; s/Ä/Ae/g; s/Ö/Oe/g; s/Ü/Ue/g")"' _ {} \;

However, it seems that find also selects all occurrences of a in a file/folder name if the ä in the -name parameter is a combined Unicode ä instead of ISO-8859-1.  For example,

$ find . -name "*[ä]*"  //<-- one letter ä
./filename_one_letter_ä
$ find . -name "*[ä]*"  //<-- combining letter ä
./filename_with_just_a
./filename_one_letter_ä
./filename_with_combining_diaeresis_ä

Therefore, the sed is passing the filenames through unchanged, and so mv complains when I ask it to rename, e.g., “Baustand” to “Baustand” (i.e., source and destination are the same).

How can I search and replace all combined Unicode ä,ö,ü,Ä,Ö,Ü in a file/folder structure on a Linux system if I am not able to search for them with find?  Is there another approach I could try?

An example of my file and directory names is:

/Projekte/03-11_Törggel_Mammern/Baustand/03-11_Törggel-Baustand_190501_0009.jpg

Which I would want to rename to

/Projekte/03-11_Toerggel_Mammern/Baustand/03-11_Toerggel-Baustand_190501_0009.jpg

The output of echo $LANG is en_US.UTF-8.

6
  • Can you provide sample hierarchy structure and the result you expect? Mar 11 at 13:08
  • Doesn't this work?: find . -name "*[ÄÖÜäöü]*" -exec bash -c 'mv {} $(echo {} | sed -e "s/ä/ae/g; s/ö/oe/g; s/ü/ue/g; s/Ä/Ae/g; s/Ö/Oe/g; s/Ü/Ue/g")' \;. On my end it works fine. Also before doing a move command it is always recommended to use echo before, like this: find . -name "*[ÄÖÜäöü]*" -exec bash -c 'echo $(echo {} | sed -e "s/ä/ae/g; s/ö/oe/g; s/ü/ue/g; s/Ä/Ae/g; s/Ö/Oe/g; s/Ü/Ue/g")' \; :)
    – Bog
    Mar 11 at 17:25
  • You’re asking a very broad question, and then revealing that you know quite a bit about how to solve it.  So I can’t tell what you’re actually asking — where are you stuck?  Is the problem that find is finding files with no non-Roman (accented) characters, and that, therefore, the sed is passing the filenames through unchanged, and so mv complains when you ask it to rename “Baustand” to “Baustand” (i.e., source and destination are the same)?  Or is the problem that find isn’t finding the files that do have special characters in their names? Mar 11 at 20:28
  • @Bog I don't see a huge difference to my attemp? @G-ManSays'ReinstateMonica' I try to clarify a little: The "core" problem is, that find is finding files with non-combined-letters when I search for combined letters. See example here: pst.innomi.net/paste/pcx5a9w6sncoxmvg3ug2guk6
    – rockZ
    Mar 12 at 9:15
  • 1
    For your information, Stack Exchange honors only one @-ping per comment (the first one), so I didn’t get notified about your response to me.  It’s better to post separate responses to different people. Mar 12 at 14:00

2 Answers 2

1

I can’t reproduce this (or even test it) ATM, but…

Well, you know that [xyz] matches x or y or z.  I guess that, when you say [äöü…] (using the combining characters), it is seeing

  • a
  • (combining) ¨
  • o
  • (combining) ¨
  • u
  • (combining) ¨

and so it finds files whose names include a, o, u or (combining) ¨, but not necessarily , or .

So try looking for them separately:

find . -depth -name "*ä*" -exec bash -c 'mv "$1" "$(echo "$1" | sed -e "s/ä/ae/g")"' _ {} ';'
find . -depth -name "*ö*" -exec bash -c 'mv "$1" "$(echo "$1" | sed -e "s/ö/oe/g")"' _ {} ';'
find . -depth -name "*ü*" -exec bash -c 'mv "$1" "$(echo "$1" | sed -e "s/ü/ue/g")"' _ {} ';'
find . -depth -name "*Ä*" -exec bash -c 'mv "$1" "$(echo "$1" | sed -e "s/Ä/Ae/g")"' _ {} ';'
find . -depth -name "*Ö*" -exec bash -c 'mv "$1" "$(echo "$1" | sed -e "s/Ö/Oe/g")"' _ {} ';'
find . -depth -name "*Ü*" -exec bash -c 'mv "$1" "$(echo "$1" | sed -e "s/Ü/Ue/g")"' _ {} ';'

(without square brackets).  Note that your echo $1 (without quotes) could fail for some filenames.  (';' is equivalent to \;; I prefer to avoid backslashes stylistically.)

Or, if you really really want to do it all in one command, try

find . -depth "(" -name "*ä*" -o -name "*ö*" -o -name "*ü*"     \
               -o -name "*Ä*" -o -name "*Ö*" -o -name "*Ü*" ")" \
       -exec bash -c 'mv "$1" "$(printf "%s" "$1" | sed -e "s/ä/ae/g; s/ö/oe/g; s/ü/ue/g; s/Ä/Ae/g; s/Ö/Oe/g; s/Ü/Ue/g")"' _ {} ';'

(printf "%s" is functionally very similar to echo, but safer.  Stylistically, I would normally use printf '%s' (with single quotes); I use double quotes here because we're inside a single-quoted ('mv …') string.)

And maybe

… "(" -iname "*ä*" -o -iname "*ö*" -o -iname "*ü*" ")" …

will work.


Also, the thing you tried initially might work if you set LANG to de_DE.UTF-8.

2
  • And, as Bog pointed out in a comment that’s been deleted: It’s a good idea to back up the directory in which you want to rename the files.  It makes recovering way easier if something goes wrong. Mar 12 at 14:18
  • Thank you very much. What you described in the first place was exactly the problem. I didn't consider that [äöü…] matches also a/ä and the combining characters. I could solve it looking for the characters seperately. Thanks.
    – rockZ
    Mar 14 at 13:16
1

The de-ASCII transliteration of uconv will do what you want. For instance with the precomposed and decomposed as well as uppercase and lowercase versions of the ä character:

$ printf '\u00c4 \u00e4 A\u0308 a\u0308\n'
Ä ä Ä ä
$ printf '\u00c4\u00e4A\u0308a\u0308' | uconv -x name
\N{LATIN CAPITAL LETTER A WITH DIAERESIS}\N{LATIN SMALL LETTER A WITH DIAERESIS}\N{LATIN CAPITAL LETTER A}\N{COMBINING DIAERESIS}\N{LATIN SMALL LETTER A}\N{COMBINING DIAERESIS}
$ printf '\u00c4\u00e4A\u0308a\u0308 \u00c4 A\u0308 \u00c4B\n' | uconv -x de-ASCII
AeaeAeae AE AE AEB

(also note the Ä changed to either AE or Ae depending on context).

So, here, you could try and convert all files with non-ASCII characters in their name with something like (in zsh):

autoload zmv
zmv -n $'(**/)(*[^\1-\177]*)' '$1$(print -rn -- $2 | uconv -x de-ASCII)'

Example:

$ touch $'\u00c4\u00e4A\u0308a\u0308'
$ touch $'St\ue9phane' $'Ste\u301phane'
$ zmv -n $'(**/)(*[^\1-\177]*)' '$1$(print -rn -- $2 | uconv -x de-ASCII)'
mv -- ÄäÄä AeaeAeae
mv -- Stéphane Stephane
mv -- Stéphane Stephane

Remove the -n (dry-run) if happy.

Or to avoid running one uconv per file in case you have millions to rename.

files=(**/*[^$'\1-\177']*)
typeset -U basenames=($files:t)
typeset -A translation
print -rNC1 -- $basenames | uconv -x de-ASCII |
  for name in $basenames; do
    IFS= read -rd '' translated && translation[$name]=$translated
  done

zmv -n $'(**/)(*[^\1-\177]*)' '$1${translation[$2]-$2}'

As an answer to your more generic question of how to find files with characters with diaeresis in their decomposed form, you only need to look for the combining diaeresis character (U+308).

So:

find . -name $'*\u0308*'

Or if it has to follow only one of AOUaou:

find . -name $'*[AOUaou]\u0308*'

For the ones in their precomposed forms, you'd need to list them all individually:

find . -name '*[ÄËÏÖÜäëïöüÿŸǕǖǗǘǙǚǛǜǞǟȪȫ΅ΐΪΫΰϊϋϔӒӓӚӛӜӝӞӟӤӥӦӧӪӫӬӭӰӱӴӵӸӹḦḧḮḯṎṏṲṳṺṻẄẅẌẍẗ⍡⍢⍣⍤⍥⍨⍩⸚]*'

Or:

find . -name $'*[\uA8\uC4\uCB\uCF\uD6\uDC\uE4\uEB\uEF\uF6\uFC\uFF\u178\u1D5\u1D6\u1D7\u1D8\u1D9\u1DA\u1DB\u1DC\u1DE\u1DF\u22A\u22B\u385\u390\u3AA\u3AB\u3B0\u3CA\u3CB\u3D4\u4D2\u4D3\u4DA\u4DB\u4DC\u4DD\u4DE\u4DF\u4E4\u4E5\u4E6\u4E7\u4EA\u4EB\u4EC\u4ED\u4F0\u4F1\u4F4\u4F5\u4F8\u4F9\u1E26\u1E27\u1E2E\u1E2F\u1E4E\u1E4F\u1E72\u1E73\u1E7A\u1E7B\u1E84\u1E85\u1E8C\u1E8D\u1E97\u2361\u2362\u2363\u2364\u2365\u2368\u2369\u2E1A]*'

Using zsh's $'\uXXXX' notation now supported by a few other shells including bash.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .