XTerm only display part of some unicodes and Urxvt displays nothing

Question

In my case, (U)XTerm does display some unicodes, but only partially.

For exmaple: for letters are: 𝒫φϕ𝒟.

Here is a screenshot of UXTerm and Urxvt respectively

It seems that the problem is related to the letter space or letter length.

By the way, everything is fine with gnome-terminal.

My current .Xresources file is:

Xft.dpi:       120
Xft.antialias: true
Xft.hinting:   true
Xft.rgba:      rgb
Xft.autohint:  false
Xft.hintstyle: hintslight
Xft.lcdfilter: lcddefault

XTerm*faceName:          Fixed
XTerm*faceSize:          14
XTerm*reverseVideo:      false
XTerm*selectToClipboard: true

UXTerm.vt100*faceName:          Fixed
UXTerm.vt100*faceSize:          14
UXTerm.vt100*reverseVideo:      false
UXTerm*selectToClipboard: true

URxvt.font:            xft:Fixed:pixelsize=18

URxvt.depth:                      32
URxvt.background:                 #222D31
URxvt*scrollBar:                  false
URxvt*mouseWheelScrollPage:       false
URxvt*cursorBlink:                true
URxvt*background:                 black
URxvt*foreground:                 grey
URxvt*saveLines:                  5000

How to make XTerm and Urxvt to display those characters correctly? Is it possible?

Answer 1

With xterm, the problem is that the first/last characters go outside the bounding-box which xterm reads from the fontconfig metrics. xterm generally uses the system's wcwidth function to get a (more) correct estimate of individual character width, and when drawing characters clips, and the erroneous pixels are lost. By the way, "fixed" is probably not an entry in fc-list output. fontconfig silently uses its default font for this. For urxvt - it's probably simply ignoring the glyph, since it doesn't follow the font-metrics. Some fonts are worse than others; fontconfig doesn't help in this area.

In some cases, the system's wcwidth is not up-to-date to handle some special characters, and xterm will unnecessarily truncate the cell. The sample

𝒫φϕ𝒟

uses U+1D4AB and U+1D49F, which xterm and the system agree should occupy one cell on the screen. UnicodeData.txt gives this information:

1D4AB;MATHEMATICAL SCRIPT CAPITAL P;Lu;0;L; 0050;;;;N;;;;;
1D49F;MATHEMATICAL SCRIPT CAPITAL D;Lu;0;L; 0044;;;;N;;;;

but refers you to the EastAsianWidth.txt file for the actual width. EastAsianWidth.txt (see TR11-38) has this information:

1D4A9..1D4AC;N   # Lu     [4] MATHEMATICAL SCRIPT CAPITAL N..MATHEMATICAL SCRIPT CAPITAL Q
1D49E..1D49F;N   # Lu     [2] MATHEMATICAL SCRIPT CAPITAL C..MATHEMATICAL SCRIPT CAPITAL D

That N means neutral: if the characters were used in an East-Asian context, an implementation could treat them as "full-width" (a misleading term for double-width). They're not used in that context. However, some fonts are (unsurprisingly) developed by East Asians, and have made most of the neutral characters in those fonts double-width. That leads to surprises. If the terminal (like xterm) clips the character, users are displeased. If the terminal uses the font-width, users are confused (because terminal programs are written for a fixed-pitch grid of rows/columns).

Correcting that problem (and making fonts useful to everyone) is what fontconfig is supposed to do. From xterm's standpoint, the problem is likely in the fontconfig metrics.

xterm does have a way to adapt to narrow/wide choices for ambiguous width characters, but changing the width of neutral characters is not a feature of xterm.

Here is a screenshot showing that gnome-terminal does not clip the characters, nor does it interpret those as double-width. It simply overwrites adjacent characters with the erroneous pixels:

The suggested link to Why doesn't xterm support double-sized characters when using Xft fonts? is for a different problem.