6

I was reading about Linux 5.2 patch note released at last year, I noticed that they started to optional support for case-insensitive names in ext4 file system.

So... what I am wondering is the reason why the case-insensitive option (including casefold and normalization) was needed in the kernel. I could find out another article written by Krisman who wrote the kernel code for supporting case-folding file system, but case-insensitive file system allows us to resolve important bottlenecks for applications being ported from other operating systems does not reach my heart and I cannot understand how the process of normalization and casefolding allow us to optimize our disk storage.

I appreciate so much for your help!

5
  • For easier porting of buggy Windows applications that don't always use the same capitalization for file names, or for running them in Wine. see wiki.winehq.org/Case_Insensitive_Filenames
    – Bodo
    Commented Sep 24, 2020 at 9:13
  • 1
    I agree it's quite unexpected especially considering Linus lashed out at other FS implementations for doing it in the past (see his HFS+ rant from a few years ago). Commented Sep 24, 2020 at 9:16
  • 4
    See also the LWN article on the topic. @user431397, didn’t the patch switch from NFKD to NFD in the end? Filenames are stored as provided anyway, not canonicalised; canonicalisation is only done for comparison. Commented Sep 24, 2020 at 9:18
  • @user431397 I’m not saying it’s good, I agree there are quite a few problems with this. I was only wondering about the accuracy of “They’re storing filenames in NFKD” — they’re not storing them in any canonical form, and they’re not using NFKD ;-). Commented Sep 24, 2020 at 20:35
  • 1
    (For more on text handling, see Dark corners of Unicode.) Commented Sep 28, 2020 at 16:14

3 Answers 3

5

case-insensitive file system allows us to resolve important bottlenecks for applications being ported from other operating systems

does not reach my heart and I cannot understand how the process of normalization and casefolding allow us to optimize our disk storage.

Wine, Samba, and Android have to provide case-insensitive filesystem semantics. If the underlying filesystem is case-sensitive, every time a case-sensitive lookup fails, Wine et al. has to scan each directory to prove there are no case-insensitive matches (e.g. if looking up /foo/bar/readme.txt fails, you have to perform a full directory listing and case-folded comparison of all files in foo/bar/* and all directories in foo/*, and /*).

There are a few problems with this:

  • It can get very slow with deeply nested paths (which can generate hundreds of FS calls) or directories with tens of thousands of files (i.e. storing incremental backups over SMB).
  • These checks introduce race conditions.
  • It's fundamentally unsound: if both readme.txt and README.txt exist but an application asks for README.TXT, which file is returned is undefined.

Android went so far as to emulate case-insensitivity using FUSE/wrapfs and then the in-kernel SDCardFS. However, SDCardFS just made everything faster by moving the process into kenel space†. It still had to walk the filesystem (and was thus IO bound), introduced race conditions, and was fundamentally unsound. Hence why Google funded† development of native per-directory case-insensitivity in F2FS and have since deprecated SDCardFS.

There have been multiple attempts in the past to enable case-insensitive lookups via VFS. The most recent attempt in 2018 allowed mounting a case-insensitive view of the filesystem. Ted Tso specifically cited the issues with wrapfs for adding this functionality, as it would at least be faster and (I believe) free of race conditions. However, it was still unsound (requesting README.TXT could return readme.txt or README.txt). This was rejected in favor of just adding per-directory support for case-insensitivity and is unlikely to ever make it into VFS††.

Furthermore, users expect case-insensitivity thus any consumer oriented operating system has to provide it. Unix couldn't supported it natively because Unicode didn't exist and strings were just bags-of-bytes. There are plenty of valid criticisms of how case-folding was handled in the past, but Unicode provides an immutable case-fold function that works for all but a single locale (Turkic, and even then it's just two codepoints). And the filesystem b-tree is the only reasonable place to implement this behavior.

AFAICT
††I emailed Krisman, the author of both the VFS-based case-insensitive lookups and per-directory case-insensitive support on EXT4 and F2FS.

0
3

Other operating systems have case insensitive filesystem.

As example: MacOS permit case-insensitive (as default) or case-sensitive. Adobe Photoshop and Adobe Lightroom doesn't work well with case-sensitive file system. This means that within Adobe programs, there are probably hardcoded paths, written in different ways (maybe "Documents" and "documents" in the different libraries, or just sometime some filters are applied (e.g. lowercase and removing spaces, which may differ from the path of the data). Nobody cared, because it just work.

So, if now you want to port a programs made for some common proprietary operating system of our epoch, either you should fix all paths, so that you have always a consistent use of filename cases, or you prefer to have a filesystem which handle these for you.

Adobe could not do it for MacOS, so expect things are much more difficult (and costly) for other vendors. See https://helpx.adobe.com/creative-suite/kb/error-case-sensitive-drives-supported.html

0

I don't know a single reason of having a case-sensitive FS: the only thing it does is that it utterly confuses the user. Microsoft developers understood it from the get go and didn't bother with a broken concept. Now, thirty years later some Linux developers have realized that case insensitivity is safer and more logical and have finally implemented it.

Why were the very first Unix filesystems case-sensitive? It might be due to the fact that it's easier on the CPU to work with them. You don't need additional functions to consume CPU cycles to verify whether a file with a similar name already exists albeit with a different capitalization (also there are alphabets other than Latin/English where implementing case-insesitivity is not trivial). Nowadays, with modern super fast CPUs it's no big deal.

3
  • 4
    It's not that people are out to get you, it's that you have an uninformed answer and a crappy attitude. I agree with you that it confuses end-users, but it was only since ~2005ish that Unicode provided a good, cross-platform solution for case-insensitive semantics.
    – Indolering
    Commented Oct 24, 2020 at 4:09
  • 1
    @ArtemS.Tashkinov For starters: NTFS is not case-insensitive, FAT and several other file systems are. I don't know why this myth keeps getting perpetuated. The Win32 layer - what most people would consider the "Windows personality" - is case-insensitive. The fact that OSX/macOS also offers to configure volumes to be handled this way suggests your remark about how end users see this has some merit. But there are more complications that arise from case-folding, which probably explains much more why. If a file name is merely an array of bytes minus some disallowed, things are simple! Commented Oct 26, 2022 at 11:57
  • @Indolering would you mind explaining what you refer to? I find the problem is quite tricky even now. Commented Oct 26, 2022 at 11:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .