Find the best font for rendering a codepoint

Question

How to find the appropriate font for rendering unicode codepoints ?

gnome-terminal find that characters like «🉃⼼😻🕲🝤» can be rendered with fonts like Symbola rather than my terminal font or the codepoint-in-square fallback (񿿿). How ?

Answer 1

Using fontconfig,

> fc-list ':charset=<hex_code1> <hex_code2>'

e.g.

> fc-list ':charset=2713 2717'

will display any font filenames containing ✓ and ✗.

To get the codepoint corresponding to the character use (for example)

> printf "%x" \'✓
2713>

This uses a somewhat obscure feature of the POSIX printf utility:

If the leading character is a single-quote or double-quote, the value shall be the numeric value in the underlying codeset of the character following the single-quote or double-quote.

Taken together,

> printf '%x' \'✓ | xargs -I{} fc-list ":charset={}"

This uses the xargs -I flag to replace {} with names from stdin. So this effectively boils down to:

> fc-list ":charset=2713"

Answer 2

This is not necessarily the best method, and it sure isn't user-friendly, but it's easy to get working: here's a Python script to do it.

Install the Python-fontconfig library. Either get it from your distribution (e.g. sudo apt-get install python-fontconfig on Debian and derivatives) or install it in your home directory (pip install --user python-fontconfig). Then you can run this script (save it as fc-search-codepoint in a directory on your PATH, e.g. typically ~/bin, and make it executable):

#!/usr/bin/env python2
import re, sys
import fontconfig
if len(sys.argv) < 1:
    print('''Usage: ''' + sys.argv[0] + '''CHARS [REGEX]
Print the names of available fonts containing the code point(s) CHARS.
If CHARS contains multiple characters, they must all be present.
Alternatively you can use U+xxxx to search for a single character with
code point xxxx (hexadecimal digits).
If REGEX is specified, the font name must match this regular expression.''')
    sys.exit(0)
characters = sys.argv[1]
if characters.startswith('U+'):
    characters = unichr(int(characters[2:], 16))
else:
    characters = characters.decode(sys.stdout.encoding)
regexp = re.compile(sys.argv[2] if len(sys.argv) > 2 else '')

font_names = fontconfig.query()
found = False
for name in font_names:
    if not re.search(regexp, name): continue
    font = fontconfig.FcFont(name)
    if all(font.has_char(c) for c in characters):
        print(name)
        found = True

sys.exit(0 if found else 1)

Example usage:

$ fc-search-codepoint 🉃⼼😻🕲🝤
$ echo $?
1

I don't have any font with all of these characters.

$ fc-search-codepoint U+1F64D
/usr/share/fonts/truetype/unifont/unifont_upper.ttf
/usr/share/fonts/truetype/unifont/unifont_upper_csur.ttf

Answer 3

Ultimately gnome-terminal uses fontconfig to (among other things):

...efficiently and quickly find the fonts you need among the set of fonts you have installed, even if you have installed thousands of fonts...

In the API documentation you can find functions to query fonts character ranges and for operations on character ranges, but the documentation is so cryptic that I never could figure out how different sets of functions relate to each other. If I needed to dive deeper I would rather look at examples of usage in other software, perhaps vte (the terminal emulation library used in gnome-terminal).

Another library in between vte and fontconfig is pango "...a library for laying out and rendering of text, with an emphasis on internationalization...". Now that I think of it, it sounds as the one to contain most of the logic you're after.

The character coverage functionality in pango is implemented by coverage maps ("It is often necessary in Pango to determine if a particular font can represent a particular character, and also how well it can represent that character. The PangoCoverage is a data structure that is used to represent that information."), but there are probably more complicated details involved in decided what glyph to render with what font. I guess VTE relies on pango to render strings with appropriate fonts while pango uses fontconfig (or other supported font backend) to find the most appropriate font based on various pieced of logic in pango itself and/or the backend.

Answer 4

I altered the code to check if a font contains all characters of a certain string. So this can be called by fc-search-codepoint "$fontname" "$string" and it returns the exit code 0 on success or 1 otherwise. The font names can be retrieved from fc-query /path/to/FontSandMonoBoldOblique.ttf or Imagemagick's convert -list font. I use it to check if a user selected string can be rendered with the user selected font and if the command fails, a fallback font is used.

#!/usr/bin/env python2
import re
import sys
import os
import fontconfig
if len(sys.argv) < 3:
    print("Usage: " + sys.argv[0] + " 'Fontname-Bold' 'String to check'")
    sys.exit(0)

font_name = sys.argv[1].decode('utf-8')
string = sys.argv[2].decode('utf-8')

if '-' in font_name:
        font_name = font_name.split('-')
        font_style = font_name[-1]
        font_name = ''.join(font_name[:-1])
else:
        font_style = ""

font_names = fontconfig.query()
for name in font_names:
    font = fontconfig.FcFont(name)
    if not len(font.family) > 0:
        continue
    for item in font.family:
        if item[1] == unicode(font_name):
            if len(font_style) == 0:
                match = "yes"
            else:
                for item in font.style:
                    if item[1] == unicode(font_style):
                        match = "yes"
            try:
                match
            except NameError:
                continue
            if all(font.has_char(c) for c in string):
                sys.exit(0)
            else:
                sys.exit(1)
print >> sys.stderr, "font not found: " + font_name + " " + font_style
sys.exit(1)