MAC OS: Cannot access folders whose names contain accented chars made of a combination of char+accent

Question

I'm trying to access from a MAC (Using Finder, or the commands 'find' or 'ls') some folders whose names contain accented chars. Some of those folders are fully visible on the MAC, some not. Here is what I did and what I found:

• I use the "lftp mirror" command to get folders from a server to my Synology NAS.
• Lftp is configured to process file names in UTF-8
• When I try to access some of those folders that contain accented chars from the MAC (using the Finder or even the commands 'ls' or 'find'), the folder is not reachable.
• Using the GUI from the NAS, I noticed that the folders in question have names with accented chars made of two different chars (When you delete an 'é' for example, you first delete the accent and then the 'e').
• Folders whose names contain simple accented chars are OK.
• When I copy those files directly to my MAC (using rsync with format conversion from UTF-8 to UTF-8 MAC), everything is OK. I even suspect I do not need to use the conversion command and simply copy the files (TBC).

So, question is 'simple': do I need to systematically convert the files before using them, or is there a better way.

Additionally, I do not want to convert the files in the directory where I am mirroring my remote server. I will need to copy the renamed files to another location to avoid issues with the mirror command.

Answer 1

I think you've probably hit a Unicode normalization issue, but I'm not quite sure how, exactly.

My guess is that rsync is decomposing composed characters for macOS, but lftp isn't. You can confirm this by doing a hex dump of an affected filename on the NAS and on your Mac. Here's an example on my Linux server:

> ls -l
total 1
-rw-rw-r-- 1 sam sam 0 May 24 11:16 NFC Pokémon
-rw-rw-r-- 1 sam sam 0 May 24 11:16 NFD Pokémon
> find . -name 'NFC*' | hexdump -C
00000000  2e 2f 4e 46 43 20 50 6f  6b c3 a9 6d 6f 6e 0a     |./NFC Pok..mon.|
0000000f
> find . -name 'NFD*' | hexdump -C
00000000  2e 2f 4e 46 44 20 50 6f  6b 65 cc 81 6d 6f 6e 0a  |./NFD Poke..mon.|
00000010

In the NFC filename “é” is represented by the byte sequence c3 a9, which is the UTF-8 encoding of the character U+00E9 LATIN SMALL LETTER E WITH ACUTE. In the NFD filename, “é” is represented by the byte sequence 65 cc 81, which is two UTF-8 characters. 65 decodes to U+0065 LATIN SMALL LETTER E, and cc 81 decodes to U+0301 COMBINING ACUTE ACCENT.

(I used find to print the filename here because I have a superstitious distrust of how shells handle filename encoding and globbing.)

Anyway, you can use this method to figure out how exactly the filenames are encoded on your NAS, as well as what exactly rsync and lftp are doing (or not doing) to the filenames when saving them to your Mac.

Regardless of what exactly is going on, I think most of the time you can probably fix things by composing all your filenames. I do this on my server on a semi-regular basis:

convmv -r -f utf-8 -t utf-8 --nfc /data/AUDIO/Library

N.B.: If you do this to files that are currently being seeded in a torrent client, it might break things. I think it depends on the client. Other software that depends on the exact byte sequences in filenames could also lose track of files you renormalize.