Quick solution:
do not use accented letters on your keyboard, use tab-complete instead (and have your SSH key setup so that tab-complete also works with over the network scp
, rsync
, etc.) or fall back to wild cards, because what you experience is the normal intended behaviour.
It doesn't work, because you did not type the same filename.
Seems crazy ? That's UTF-8 to you.
Even more crazy: I can use my magical remote mind-reading psychic power to tell you that you have an Apple Mac.
More seriously: that's the crucial information you forgot to give when asking your question, but that you accidentally leaked when typing the question itself.
While copy-pasting the answer above:
# echo "scp [email protected]:/home/me/cömmön_file.jpg" | hexdump -C
00000000 73 63 70 20 6d 65 40 65 78 61 6d 70 6c 65 2e 63 |scp [email protected]|
00000010 6f 6d 3a 2f 68 6f 6d 65 2f 6d 65 2f 63 6f cc 88 |om:/home/me/co..|
00000020 6d 6d 6f cc 88 6e 5f 66 69 6c 65 2e 6a 70 67 20 |mmo..n_file.jpg |
00000030 2f 68 6f 6d 65 2f 6d 65 2f 0a |/home/me/.|
0000003a
Please pay close attention to how the letter 'ö' is coded : 6f cc 88
. A litteral 'o' followed by an extra UTF-8 codepoint. (in fact, on my terminal it doesn't even display as 'ö' but as 'o')
When when I (=Linux user) type:
echo /home/me/cömmön_file.jpg | hexdump -C
00000000 2f 68 6f 6d 65 2f 6d 65 2f 63 c3 b6 6d 6d c3 b6 |/home/me/c..mm..|
00000010 6e 5f 66 69 6c 65 2e 6a 70 67 0a |n_file.jpg.|
0000001b
Again look closely at the 'ö' symbol : c3 b6
, an entirely different UTF-8 code point and no extra litteral ASCII.
Ultra short explanation : UTF-8 normalization (composition vs decomposition).
Longer explanation :
in Unicode, there are multiple way to code for something that looks like 'ö'.
- first way is composed characters : there's a code point that's litteraly 'ö' inherited from Latin-1 (ISO/IEC 8859-1:1998) code points, Unicode codepoint U+00f6 (coded as c3 b6 in UTF-8)
- second way is decomposed characters : you first output the ASCII o, and then append a special code point that means 'Please combine an umlaut to the preceding letter', Unicode codepoint U+0308 (coded as cc 88 in UTF-8)
it's this combining character that enable you to do all the̫ ͨcra̎zy shit̫ ĺiͭke̬̓ ̭Z͉̒a̅l̞gͩoͤ ̤͋aṅd̲ ̹ͨallͦ ̍ͅthͅe oͅt͔̅h̦̊e̠r ͔̋dḁŕ͕k̓ ̃m͍o͉ͅñ͎͖̉s̺͑tr̰͎̈́ỏ͖ͧsi̮͂͑t̚i͙̗ės͓̊̒ ̞ͯt̗͕ẖ̈ͩá̝ṱ̟͒ ͓͐ͦl̈́ṵ̿r͈̾k̼̝ͭ̍ ̹i͖̇̈́n͚̳ ͖̗ͦt͓h̿e͖ ̌m̳͌̽a̪ͥd̺͑n͕͌̐e̿͊s͇s̘͓͊ ̗̈́ö̫́f͕̞ ͕̰̓ìṅ̠sͤ̂a̬̝̿ͪn̘ͫ͆e̜ͯ ̩͓ͣẻ͛ḽ̞̃ḓ̺r̙̦ͥͬi̫̠̔ͮt̰̓̾ͅč͕ͦḧ̞̱͖́̒̽ ͇̳ḁ̖̊̈b̏͑o̳̙̍m̩̪̞ͦi̇ͮn̳͔ͨ̏ͤa̤̯ͣṱ̰ͥï̺̄o̞͖̿n͆ͦs̬̍ ̹ͩ͒th̞̄a̗̗͐͌ͪt͂ ̬̞iͭ̒s̘͇ ̱̯̐̆̒Ũ̺̞̘ͯT̩̀̔̚F̪͒̄-̪̘̈́8̮̆̍͂.̱͍̂
hum.
The rest of the planet uses composed characters whenever possible (because it's more compact and also because it uses the range of Unicode that is compatible with Latin-1, simplifying backward compatibility) and only resort to combining characters for thing that don't have their own code point (mostly less frequent languages).
Apple lives apparently on a different planet, and they have decided that they try to always use combining characters (because they worship the dark lord Za͓̙̘͌l̦̖͉̃ͦ͆͊ͧ̀g͖̭̼̗͉̦̬̍̀̌ͬ̓ͥ҉o̧͉̗̱̥̣̯͍̗̲̩ͪ͋̾͑̈́ͦ̐̓͘͡ ?).
Typing the keyboard letter that looks like 'ö' simply doesn't generate the same binary sequence depending on which computer you type the key.
Then comes into play another thing : most Unix tend to use file systems (like Linux' EXT4) which are sensitive to case AND sensitive to Unicode coding (where UTF-8 is supported). They try to preserve whether the text was composed or not. Thus they make a distinction between the UTF-8 binary sequence 6f cc 88
and c3 b6
even if they code for the same end result 'ö'. (the same way the make a distinction between 'A' and 'a' even if its the same latin letter).
So your 'ö' produced by your keyboard and the 'ö' on the server are not the same.
It happens that stack exchange just store whatever Unicode coding you throw at it as-is, leading to mythical answers as the HTML RegEx parser ones. (Thus your Mac betrayed itself by the specific byte sequence that recorded 'ö').
locale
) This type of problems might happen when both side don't use the same locale (preferably a.UTF-8
ending one).rsync
instead ofscp
also barf on that file ?