14

How to find the appropriate font for rendering unicode codepoints ?

gnome-terminal find that characters like «🉃⼼😻🕲🝤» can be rendered with fonts like Symbola rather than my terminal font or the codepoint-in-square fallback (񿿿). How ?

14

This is not necessarily the best method, and it sure isn't user-friendly, but it's easy to get working: here's a Python script to do it.

Install the Python-fontconfig library. Either get it from your distribution (e.g. sudo apt-get install python-fontconfig on Debian and derivatives) or install it in your home directory (pip install --user python-fontconfig). Then you can run this script (save it as fc-search-codepoint in a directory on your PATH, e.g. typically ~/bin, and make it executable):

#!/usr/bin/env python2
import re, sys
import fontconfig
if len(sys.argv) < 1:
    print('''Usage: ''' + sys.argv[0] + '''CHARS [REGEX]
Print the names of available fonts containing the code point(s) CHARS.
If CHARS contains multiple characters, they must all be present.
Alternatively you can use U+xxxx to search for a single character with
code point xxxx (hexadecimal digits).
If REGEX is specified, the font name must match this regular expression.''')
    sys.exit(0)
characters = sys.argv[1]
if characters.startswith('U+'):
    characters = unichr(int(characters[2:], 16))
else:
    characters = characters.decode(sys.stdout.encoding)
regexp = re.compile(sys.argv[2] if len(sys.argv) > 2 else '')

font_names = fontconfig.query()
found = False
for name in font_names:
    if not re.search(regexp, name): continue
    font = fontconfig.FcFont(name)
    if all(font.has_char(c) for c in characters):
        print(name)
        found = True

sys.exit(0 if found else 1)

Example usage:

$ fc-search-codepoint 🉃⼼😻🕲🝤
$ echo $?
1

I don't have any font with all of these characters.

$ fc-search-codepoint U+1F64D
/usr/share/fonts/truetype/unifont/unifont_upper.ttf
/usr/share/fonts/truetype/unifont/unifont_upper_csur.ttf
  • 1
    Thats a very helpful script! However, it is only python2 compliant, and i suppose it is a bit nasty to do exactly that portable. Would you mind at least changing the #!/usr/bin/env python to #!/usr/bin/env python2 as per PEP 394. – Zulan Jun 6 '16 at 8:38
  • 1
    Thanks for this answer! It was very helpful. I'm sure the OS or system libraries that implement font fallback are doing something more efficient, but this works. @Zulan It can be made to work with python3 too; I just wrote a smaller version of this at the bottom of this answer. – ShreevatsaR Jul 18 '17 at 4:13
3

Ultimately gnome-terminal uses fontconfig to (among other things):

...efficiently and quickly find the fonts you need among the set of fonts you have installed, even if you have installed thousands of fonts...

In the API documentation you can find functions to query fonts character ranges and for operations on character ranges, but the documentation is so cryptic that I never could figure out how different sets of functions relate to each other. If I needed to dive deeper I would rather look at examples of usage in other software, perhaps vte (the terminal emulation library used in gnome-terminal).

Another library in between vte and fontconfig is pango "...a library for laying out and rendering of text, with an emphasis on internationalization...". Now that I think of it, it sounds as the one to contain most of the logic you're after.

The character coverage functionality in pango is implemented by coverage maps ("It is often necessary in Pango to determine if a particular font can represent a particular character, and also how well it can represent that character. The PangoCoverage is a data structure that is used to represent that information."), but there are probably more complicated details involved in decided what glyph to render with what font. I guess VTE relies on pango to render strings with appropriate fonts while pango uses fontconfig (or other supported font backend) to find the most appropriate font based on various pieced of logic in pango itself and/or the backend.

3

Using fontconfig,

> fc-list ':charset=<hex_code1> <hex_code2>'

e.g.

> fc-list ':charset=2713 2717'

will display any font filenames containing ✓ and ✗.

To get the codepoint corresponding to the character use (for example)

> printf "%x" \'✓

Taken together,

> printf '%x' \'✓ | xargs -I{} fc-list ":charset={}"
  • 2
    Note that you need a version of fontconfig that is 2.11.91 or later. – Nathaniel M. Beaver Jan 11 '18 at 1:49
  • 1
    note that dash printf and /bin/printf dont support that – Steven Penny Jan 13 '18 at 19:53
  • 1
    Awesome! I've been looking for info on this for a long time. Note that you can also specify ranges as well as single characters, so to find all fonts that have all the box-drawing characters, for example: fc-list --format='%{postscriptname}\n' ':charset=2500-257F' – Neil Mayhew Sep 21 '18 at 7:15
1

I altered the code to check if a font contains all characters of a certain string. So this can be called by fc-search-codepoint "$fontname" "$string" and it returns the exit code 0 on success or 1 otherwise. The font names can be retrieved from fc-query /path/to/FontSandMonoBoldOblique.ttf or Imagemagick's convert -list font. I use it to check if a user selected string can be rendered with the user selected font and if the command fails, a fallback font is used.

#!/usr/bin/env python2
import re
import sys
import os
import fontconfig
if len(sys.argv) < 3:
    print("Usage: " + sys.argv[0] + " 'Fontname-Bold' 'String to check'")
    sys.exit(0)

font_name = sys.argv[1].decode('utf-8')
string = sys.argv[2].decode('utf-8')

if '-' in font_name:
        font_name = font_name.split('-')
        font_style = font_name[-1]
        font_name = ''.join(font_name[:-1])
else:
        font_style = ""

font_names = fontconfig.query()
for name in font_names:
    font = fontconfig.FcFont(name)
    if not len(font.family) > 0:
        continue
    for item in font.family:
        if item[1] == unicode(font_name):
            if len(font_style) == 0:
                match = "yes"
            else:
                for item in font.style:
                    if item[1] == unicode(font_style):
                        match = "yes"
            try:
                match
            except NameError:
                continue
            if all(font.has_char(c) for c in string):
                sys.exit(0)
            else:
                sys.exit(1)
print >> sys.stderr, "font not found: " + font_name + " " + font_style
sys.exit(1)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.