Detect how much of Unicode my terminal supports, even through screen

Question

Here's the problem: I want to be able to discern if my terminal is capable of decent unicode or not, in order to use some characters or not, much like glances does, that sometimes uses colors and others underline.

The motivation arises because in any kind of virtual terminal I get decent fonts, but I understand that the basic Linux console has a character set of 256 or 512 simultaneous symbols, so you cannot expect full font support.

At first I thought that I could use $TERM or tty, but here's the catch: I'm using byobu too, so $TERM is always "screen.linux". The output of tty is also not very telling: /dev/pts/<some number> in both "real" and virtual terms.

$BYOBU_TTY is no help either, because e.g. it may be /dev/tty1 and when the session is opened in Ctrl+Alt+F1 the characters don't show but when attaching to the same session from some X term, they show properly and still $BYOBU_TTY does not change. Besides, I'd like to be able to detect this without presuming byobu is there or not.

Also, locale shows in all cases en_US.UTF-8

Yet somehow glances (to name a particular tool I see detecting this), even inside byobu, uses different output depending on the terminal I'm attaching to the byobu session.

I'm having trouble with google because terminal and tty seem too common search terms. At most I arrive at solutions recommending $TERM or tty.

Answer 1

Well, first I guess I would point out that pretty much all terminals these days are "virtual" in the sense you talk about... even if the terminal is at the other end of a bona fide serial port. I mean, the days of VT-100s, Wyse terminals and other "physical", "real" terminals are pretty much gone!

That aside, let's say you want to detect what kind of Unicode support your terminal has. You can do this by writing test characters to is and seeing what happens. (You can make an effort to erase the test characters after you've written then, but the user may still see them briefly, or erasing them might not work properly in the first place.)

The idea is to ask the terminal to tell you its cursor position, output a test character, ask the terminal again to tell you its position, and compare the two positions to see how far the terminal's cursor moved.

To ask the terminal for its position, see here. Essentially:

echo -e "\033[6n"; read -d R foo; echo -en "\nCurrent position: "; echo $foo | cut -d \[ -f 2

Try outputting "é". This character takes 2 bytes in UTF-8 but displays in only one column on the screen. If you detect that outputting "é" causes the cursor to move by 2 positions, then the terminal has no UTF-8 support at all and has probably output some kind of garbage. If the cursor didn't move at all, then then terminal is probably ASCII only. If it moved by 1 position, then congratulations, it can probably display French words.

Try outputing "あ". This character takes 3 bytes in UTF-8 but displays in only two columns on the screen. If the cursor moves by 0 or 3, bad news, similar to above. If it moves by 1, then it looks like the terminal supports UTF-8 but doesn't know about wide characters (in fixed-width fonts). If it moves by 2 columns, all is good.

I'm sure there are other probe characters that you could emit which would lead to useful information. I am not aware of a tool that does this automatically.

Answer 2

OP's actual question is: what Unicode values does the Linux console support, and can those be detected while running screen. In principle, one can do this by retrieving the Unicode map for the console.

The kbd source tree contains getunimap (and its manual page). The manual page says that

The getunimap program is old and obsolete. It is now part of setfont

which is not exactly true. setfont has an option which does roughly the same thing:

   -ou file                                  
          Save previous Unicode map in file

The differences:

• setfont writes to a file, while getunimap writes to the standard output
• getunimap shows the character which would be mapped, as a comment.

For example:

0x0c4   U+2500  # ─ 
0x0c4   U+2501  # ━ 
0x0b3   U+2502  # │ 
0x0b3   U+2503  # ┃ 
0x0da   U+250c  # ┌ 
0x0da   U+250d  # ┍ 
0x0da   U+250e  # ┎ 
0x0da   U+250f  # ┏ 
0x0bf   U+2510  # ┐ 
0x0bf   U+2511  # ┑ 
0x0bf   U+2512  # ┒ 
0x0bf   U+2513  # ┓ 
0x0c0   U+2514  # └ 
0x0c0   U+2515  # ┕ 
0x0c0   U+2516  # ┖ 
0x0c0   U+2517  # ┗ 

versus

0xc4    U+2500
0xc4    U+2501
0xb3    U+2502
0xb3    U+2503
0xda    U+250c
0xda    U+250d
0xda    U+250e
0xda    U+250f
0xbf    U+2510
0xbf    U+2511
0xbf    U+2512
0xbf    U+2513
0xc0    U+2514
0xc0    U+2515
0xc0    U+2516
0xc0    U+2517

If you are running in screen (or for example running xterm and not on the console), you will get a permissions error which you can work around using sudo.

If I happen to know which font was loaded, I can check that (without special permissions) using psfgettable, e.g.,

zcat /usr/share/consolefonts/Lat2-Fixed16.psf.gz | psfgettable -

and see the mapping data which setfont would use to load the font (with the Unicode mapping):

#
# Character table extracted from font -
#
0x000   U+00a9
0x001   U+00ae
0x002   U+00dd
0x003   U+0104
0x004   U+2666 U+25c8 U+fffd
0x005   U+0105
0x006   U+0111
0x007   U+0150
0x008   U+0151
0x009   U+0162
0x00a   U+0164
0x00b   U+0170
0x00c   U+0171
0x00d   U+021a 
0x00e   U+02dd  
0x00f   U+2014 U+2015
0x010   U+2020
0x011   U+2021
0x012   U+2022 U+25cf
...

Both getunimap and setfont give the data unsorted, while psfgettable appears to be sorted (as well as combining lines for Unicode values that map to the same glyph). So there are differences, but the information is accessible.

Further reading (illustrating why you cannot use showconsolefont to solve this problem):

• Why does showconsolefont have different output in tmux?
• Print out binary data as is without breaking the terminal

Answer 3

I came across this question as I was trying to accomplish the same thing, but did not want to leave anything on the screen and have it set a variable, so I put the following in a shell script that I source:

function test_unicode {
  echo -ne "\xe2\x88\xb4\033[6n\033[1K\r"
  read -d R foo
  echo -ne "\033[1K\r"
  echo -e "${foo}" | cut -d \[ -f 2 | cut -d";" -f 2 | (
    read UNICODE
    [ $UNICODE -eq 2 ] && return 0
    [ $UNICODE -ne 2 ] && return 1
  )
}

test_unicode
RC=$?
export UNICODE_SUPPORT=`[ $RC -eq 0 ] && echo "Y" || echo "N"`
unset test_unicode