1

Title says it all: if I have the bytes of a file path and I try to decode them as UTF-8 is it possible that the operation will fail (i.e. the bytes of a valid unix file path can be invalid UTF-8)?

0

2 Answers 2

5

In addition to Artem S. Tashkinov's answer:

A common reason to stumble over invalid UTF-8 sequences within filenames: They've been created under a different locale setting. And yes: It's not uncommon to find systems still running with an ISO-8859 character set encoding. Not to mention the myriads of different character sets used on PCs and MACs all over the world...

The following example shows two files with the same name Überraschung (german for surprise) in the same directory. One file was created with LANG=en_us.UTF-8, the other one with LANG=en_US.ISO-8859-15:

when listed using the UTF-8 encoding:

    $ ls -lt
    total 0
    -rw-r--r-- 1 ktf ktf 0 Sep  6 12:01 ''$'\334''berraschung'
    -rw-r--r-- 1 ktf ktf 0 Sep  6 12:00  Überraschung

and when listed using the ISO-8859-15 encoding:

    $ ls -lt
    total 0
    -rw-r--r-- 1 ktf ktf 0 Sep  6 12:01  Überraschung
    -rw-r--r-- 1 ktf ktf 0 Sep  6 12:00 'Ã'$'\234''berraschung'

Hint: To properly switch between character sets you have to change both of the following settings:

  1. The setting of the environment variable LANG within the UNIX/Linux shell. That controls how the UNIX-system interprets and handles the characters.
  2. The character encoding of your terminal emulation (e.g. gnome-term or putty). That controls how the characters are displayed on your screen.
2

Yes, perfectly. The file path/name for native Linux filesystems can contain any binary data aside from NULL and /.

touch "$(dd if=/dev/urandom bs=128 count=1)"

will get you there. The command may fail since it doesn't sanitize anything - you may need to run it several times for good measure.

2
  • Please don't let beginners create files named like * or - they'll try to delete later!
    – U. Windl
    Sep 6, 2021 at 22:47
  • LMAO :-) ...... Sep 7, 2021 at 2:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.