Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

First off: Although I can understand the relevance of geographic-region detail for LC_TIME, LC_NUMERIC, and most other LC_* vars, I don't quite see how 'en_AU' relates to LC_CTYPE...
Isn't UTF-8 (and other encodings) enough of a definition in themselves (for LC_CTYPE), as and encoding by definiton is consistant?...

Thinking about it as I write, it may be that different regions of the world capitalize their lower-case letters differently... If this is the case, how does iconv handle this?..
This iconv point is actually what started me on this line of thought, becaue it doesn't ask for a locale, it only asks for the input encoding format.

My next puzzle is: What do the line-items in the output from locale LC_CTYPE refer to, and/or where is a good place to get a layout... Perhaps a more relevant question is: By whom, and where, would this info be needed? ..
I'm pretty sure I don't need it... but it all help to fill in the picture of 'scripts' and 'encodings' and 'locales'; which is surprisingly non-trivial as soon as you leave the world of ASCII.

share|improve this question

1 Answer 1

up vote 6 down vote accepted

All the locale variables use the same locale name so that you can specify your favorite locale in a single swoop, e.g. LANG=en_AU.utf8. As you surmise, the country information is occasionally relevant even in LC_CTYPE, e.g. the uppercase version of i is I in most languages but İ in Turkish (tr_TR.utf8). But don't expect miracles; for example the lowercase-uppercase correspondence is one-to-one, so there's no good uppercase version of ß in de_DE.iso8859-1 (it should be SS).

You'll have an easier time understanding the output of locale -k LC_CTYPE, with -k to see the keyword names in addition to the values (without -k, the output format is designed so you can get the value of a specific keyword, e.g. locale ctype-width). The list of keywords and their meanings is system-dependent, as is the way locale data is stored, and doesn't interest many people, so you may not find much documentation outside the source code of your C library. By far the most useful form of the locale command is locale -a to list available locale names.

For GNU libc (i.e. non-embedded Linux):

  • All locale data other than messages is stored in /usr/lib/locale/locale-archive. This file is generated by localedef from data in /usr/share/i18n and /usr/local/share/i18n. The format of the locale definition files in /usr/share/i18n/locales is only documented in the source code, I think.
  • The format of the character set and encoding definition files in /usr/share/i18n/charmaps is standardized by POSIX:2001. These files (or, in GNU libc, the compiled version in /usr/lib/locale/locale-archive) are used by the iconv programming and commmand line facility. Encoding conversions also rely on code in /usr/lib/gconv/*.so. The Gnu libc manual documents how to write your own gconv module, though that section contains the text “This information should be sufficient to write new modules. Anybody doing so should also take a look at the available source code in the GNU C library sources.”.
  • Message catalogs get special treatment because each application comes with its own set. Message catalogs live in /usr/share/locale/*/LC_MESSAGES. The manual contains documentation for application writers. GNU libc supports both the POSIX interface catgets and the more powerful gettext interface.

Written languages are indeed very complicated, even if you don't stray far from English. Are the French and German ü the same character (is a “tréma” exactly the same as an “umlaut”, and does it matter that French and German printers typeset the accent at a slightly different height)? What is the uppercase of i (it's İ in Turkish)? Does Ö transliterate to O if you only have ASCII (in German, it's OE)? Where is Ä sorted in a dictionary (in Swedish, it's after Z)? And that's just a few examples with European languages written in the latin alphabet! The Unicode mailing list has a lot of examples and sometimes heated discussions on such topics.

share|improve this answer
    
thanks :).. understood.. locale does it's best in very difficult terrain, and anyone dealing with a fringe case would (hopefully) be aware of the quirks involved and make special concessions... The locale ctype-width example solved my real-world problem of how to generically get iconv to print out the Unicode Codepoint of a particular char based on the current locale (for the Multi-Lingual-Plane..UTF-16 surrogates as usual being the fly in the ointment ).. I think(?) this should do the trick: echo -n "∴" |iconv -f "$(locale charmap)" -t "UTF-16BE" |xxd -p ... 2234 –  Peter.O Apr 26 '11 at 15:51
    
I've just noticed, that en_AU (in en_AU.UTF-8) etc, appears to be just a name.. It is the name of a file which defines that particular "whatever-name-you-like" locale... but that now makes me wonder where the .UTF-8 suffix comes from, because all the LC_* vars are defined in this file... Perhaps locale gets all its info from this file, either directly or indirectly, and just tacks on the '.UTF-8` suffix, but I can't see any reference to UTF-8.. These definition files are in `/usr/share/i18n/locales" –  Peter.O Apr 26 '11 at 17:11
    
@fred.bear: Try strace date and other commands to see what files are actually used. Most locale information is compiled and not read from /usr/share/i18n/locales under normal operation. See my edit. –  Gilles Apr 26 '11 at 20:03
    
thanks again... I think this may explain to some degree where the .UTF-8 suffix comes from... I just changed my default date by modifying /usr/share/i18n/locales/en_AU... using this command: sudo localedef -f UTF-8 -i en_AU_mod fred ... The default output of date is now [2011-04-27][11.21.44] and the output from locale |grep LC_TIME is: LC_TIME=fred ... :) –  Peter.O Apr 27 '11 at 1:25

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.