3

I am in the process of setting up a ZFS file server. I am planning to store all sorts of files in it. I have several hundred music tracks that are in Japanese/Korean native character.

I want the file server to retain this native character naming. Do I need to setup my ZFS pool with utf8only and/or normalization options?

e.g.

copy japanese songs into file server (via mounted shared folder in Windows). If I view the files on my file server in Windows Explorer, they must be displayed in their native characters.

Thanks a lot.

2

ZFS currently has a 255 bytes limit on names. If you decide to use UTF-16 names, you will get a limit of 127 characters. UTF-8, OTOH might encode japanese/korean in 3-byte character, limiting you even further (255/3 = 85).
I'm not aware of other concerns, aside from slower indexing of UTF-8 3-byte char strings compared to UTF-16 2-byte char strings.

So - if all names are in UTF8 - go ahead and set the 2 properties. If you use UTF16, don't.

Notice that UTF8 has specialized handling in code (zol,not *bsd, but codebase is the same). I would probably set utf8only to force conversions to utf8, and normalization to allow proper comparison of file names.

  • Could you provide a link to documentation about the limit, please? I haven't been able to find anything definitive in my (admittedly none-too-thorough) searches... Thanks! – D_Bye Jul 22 '14 at 11:20
  • not documentation, just wikipedia - en.wikipedia.org/wiki/Comparison_of_file_systems#Limits – Dani_l Jul 22 '14 at 11:49
  • 1
    Can ZFS store file names in UTF-16? If so, what are they converted to in the driver? Unix systems pass file names around as byte strings not containing null bytes, which precludes UTF-16 with 16-bit words split as two bytes. – Gilles Jul 22 '14 at 22:43
  • @Gilles I don't know what the target stream looks like. ZFS has it's roots in solaris, and the conversion is supposedly handled by the solaris uconv, instead of linux iconv, and see the uconv functions in the spl (solaris portability/compatibility layer which is deployed with zfs on linux) fossies.org/dox/zfs-0.6.3/uconv_8c.html – Dani_l Jul 23 '14 at 10:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.