Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Does it depend on what file system I use? For example, ext2/ext3/ext4 but also what happens when I insert one of those "joliet" CD-ROMs with ISO 9660? I've heard that POSIX contains some sort of spec for the charset encoding of filenames?

Essentially, what I wonder is if I got a UTF-8 encoded filename, what processing/coversion do I need to do before I pass it to a file I/O API in Linux?

share|improve this question
add comment

3 Answers

As noted by others, there isn't really an answer to this: filenames and paths do not have an encoding; the OS only deals with sequence of bytes. Individual applications may choose to interpret them as being encoded in some way, but this varies.

Specifically, Glib (used by Gtk+ apps) assumes that all file names are UTF-8 encoded, regardless of the user's locale. This may be overridden with the environment variables G_FILENAME_ENCODING and G_BROKEN_FILENAMES.

On the other hand, Qt defaults to assuming that all file names are encoded in the current user's locale. An individual application may choose to override this assumption, though I do not know of any that do, and there is no external override switch.

Modern Linux distributions are set up such that all users are using UTF-8 locales and paths on foreign filesystem mounts are translated to UTF-8, so this difference in strategies generally has no effect. However, if you really want to be safe, you cannot assume any structure about filenames beyond "NUL-terminated, '/'-delimited sequence of bytes".

(Also note: locale may vary by process. Two different processes run by the same user may be in different locales simply by having different environment variables set.)

share|improve this answer
add comment

The unix/posix layer of linux doesn't care which encoding you use. It stores the byte sequence of your current encoding as-is.

I think those mount options are there to help you convert specific filesystems that define a charset to your system charset. (CDROMs, NTFS and the FAT variants use some unicode variants).

I wish unix defined a system global encoding, but it is actually a per user setting. So if you define a different encoding then your collegue, your filenames will show up differently.

share|improve this answer
    
Ok so then I should probably check what locale the user is currently using and convert to that for new files so that he will see the filename correctly in Nautilus etc. How can I tell what the current filename charset is for the current user? –  martin Sep 15 '10 at 18:28
add comment

It depends on how you mount the file system, just take a look at mount options for different file systems in man mount. For example iso9660, vfat and fat have iocharset and utf8 options.

share|improve this answer
    
So if I mount it using utf8 should I also pass utf8 to the open() syscall? –  martin Sep 15 '10 at 17:11
    
Also I found this ( library.gnome.org/devel/glib/unstable/… ) which seems to indicate that the charset encoding of filenames is dependent on what locale is set? –  martin Sep 15 '10 at 17:11
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.