I have a hard time understanding how the file name encoding works. On unix.SE I find contradicting explanations.
File names are stored as characters
To quote another answer: Several questions about file-system character encoding on linux
[…] as you mention in your question, a UNIX file name is just a sequence of characters; the kernel knows nothing about the encoding, which entirely a user-space (i.e., application-level) concept.
If file names are stored as characters, there has to be some kind of encoding involved, since finally the file name has to end up as a bit or byte sequence on the disk. If the user can choose any encoding to map the characters to a byte sequence that is fed to the kernel, it is possible to create any byte sequence for a valid file name.
Assume the following: A user uses a random encoding X, which translates
foo into the byte sequence α and saves it to disk. Another user
uses encoding Y. In this encoding α translates to
/, which is not
allowed as a file name. However, for the first user the file is valid.
I assume that this scenario cannot happen.
File names are stored as binary blobs
To quote another answer: What charset encoding is used for filenames and paths on Linux?
As noted by others, there isn't really an answer to this: filenames and paths do not have an encoding; the OS only deals with sequence of bytes. Individual applications may choose to interpret them as being encoded in some way, but this varies.
If the system does not deal with characters, how can particular characters
NULL) be forbidden in file names? There no notion of a
without an encoding.
An explanation would be that file system can store file names containing any
character and it's only the user programs that take an encoding into account
that would choke on file names containing invalid characters. That, in turn,
means that file systems and the kernel can, without any difficulty, handle
file names containing a
I also assume that this is wrong.
Where does the encoding take place and where is the restriction posed of not allowing particular characters?