1

I'm writing an ansi-compatible terminal emulator, and I'm implementing different control codes in the order I encounter them. One oddity while using lynx is a plethora of DC1 / XON control codes (0x11). But they don't actually intend to be those control codes. I noticed in LXTerminal that they are rendered with an arrow glyph, and the layout of the whole page in lynx makes it apparent that they intend it to work this way.

Is there a well-known spec or reference implementation that specifies 0x11 as a printable character instead of DC1?

Edit:

This circled arrow is what xterm outputs for that code, but I'm not sure why, when my encoding is UTF-8, DC1 should be output that way.

enter image description here

  • Do you have a repeatable way you can show us where lynx outputs DC1's? @thomasdickey is lynx's maintainer and can identify whether it's a bug. – Mark Plotnick Jan 30 '16 at 22:33
  • Yes, but I don't think it's a bug. It's used to render an arrow in a place where the arrow makes sense, so it seems intentional. Specifically, when you enter enough characters into an input and it scrolls a portion of it out of view, an arrow is placed on the left to indicate that it is scrollable. – Jacob Jan 31 '16 at 5:50
  • Interesting. On my Ubuntu 14.04.3 system, using TERM=xterm and LANG=en_US.UTF-8, in that situation lynx outputs the byte sequence \xe2\x86\x90, which is U+2190 (LEFTWARDS ARROW). What OS are you using, and can you show the value of TERM and the output of locale? – Mark Plotnick Jan 31 '16 at 13:42
  • I'm using ansi for my terminal type, and I verified that TERM is set to that. My locale is the same as yours. – Jacob Jan 31 '16 at 17:42
  • Oh, I'm also on Ubuntu, but version 15. – Jacob Jan 31 '16 at 17:50
2

If you are seeing controlQ (0x11) and controlS (0x13), those are XON/XOFF: artifacts of flow control. You should only be seeing those if your connection says it relies upon software flow control; the terminal driver could send those to the device (just as a user could press those keys to control the other direction).

Further reading:

The comment about "layout" for Lynx is obscure:

  • 0x11 is not a whitespace character in any character set that comes to mind.
  • looking at the character-sets that Lynx knows about, none show 0x11 as a non-printing character.
  • the display character set is configurable in the Options Menu.
  • Lynx would only be using that as a printable character due to some odd combination of locale settings and display-character set.

Lacking some details about why Lynx might be using 0x11 as a non-printing character, it sounds as if the terminal driver is sending XON/XOFF with a fairly small buffer.

Following up on the comments (which terminal is using TERM=ansi? the screenshot shows xterm), lynx can write (using curses) a left-arrow, e.g., the ACS_LARROW character. In the source code, that looks like this:

    /*
     * Draw the left scrolling-indicator now, to avoid the complication of
     * overwriting part of a multicolumn character which may lie in the first
     * position.
     */
    if (IsPanned && lft_cells) {
        CTRACE_EDIT((tfp, "Draw left scroll-indicator\n"));
        TmpStyleOn(prompting ? s_prompt_edit_arr : s_aedit_arr);
        LYmove(StartY, StartX);
        LYaddch(ACS_LARROW);
        TmpStyleOff(prompting ? s_prompt_edit_arr : s_aedit_arr);
        lft_shift = 1;
    }

Now... lynx itself knows only about this definition in <curses.h>:

#define ACS_LARROW      NCURSES_ACS(',') /* arrow pointing left */

while the terminal description for "ansi" gives the actual mapping in klone+acs:

acsc=+\020\,\021-\030.^Y0\333`\004a\261f\370g\361h\260j\331k\277l\332m\300n\305o~p\304q\304r\304s_t\303u\264v\301w\302x\263y\363z\362{\343|\330}\234~\376,

Fortunately (for readability), the mapping you are interested in is the second entry mapping comma (escaped) to octal 021:

    \,\021

and that's your DC1 (021 octal is 0x11 hexadecimal). But that mapping is not used in xterm. The line-drawing characters for xterm (unless using UTF-8, where ncurses ignores the terminal description) do not include the left-arrow. That mapping is defined in xterm-basic, which you can see has no comma (the comma on the end is a separator):

acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,

The xterm manual suggests useful values for TERM, and the "ansi" one (though on the list) is not the first. It's been there a while (predating ncurses):

   Terminal  database (terminfo (5) or termcap (5)) entries that work with
   xterm include

          an optional platform-specific entry ("xterm-new"),
          "xterm",
          "vt102",
          "vt100",
          "ansi" and
          "dumb"

For instance, I see it in X11R5 (1993). But it was introduced earlier, appearing in X10R4 (1986). I suspect the meaning of "ansi" for terminal emulators has changed slightly in 30 years (it corresponds to ANSI.SYS, which differs from xterm and so forth). Use "xterm" (or "xterm-new"), and you will get better results.

  • Sure, I'm not asking what those are according to the standard but why other terminals render an arrow, and lynx is expecting them to. – Jacob Jan 30 '16 at 20:28
  • There's never any XOFF either, so I really doubt it's trying to actually use these for XON. – Jacob Jan 30 '16 at 20:29
  • There's not enough information in the comment about layout to guess what you are seeing. Locale might be a factor. – Thomas Dickey Jan 30 '16 at 20:30
  • Other terminals are just printing it like a printable character, so the cursor advances when they're output. The rest of the web page is laid out in such a way where it obviously expects the character advancement, and the arrow is appropriate for the UI, so it's clearly expecting a rendered arrow glyph. – Jacob Jan 30 '16 at 20:36
  • xterm ignores DC1. – Thomas Dickey Jan 30 '16 at 20:37
1

I have found a web page where the control characters are given a printable representation matching your findings:

http://xhelmboyx.tripod.com/formats/latin-mac-charset.txt

 1         0x01     SOH start head             white smiley
 2         0x02     STX start text         ->| or black smiley
 3         0x03     ETX end text              |<- or heart
                                               _
 4         0x04     EOT end transmit           ^ or diamond
 5         0x05     ENQ enquiry              up arrow or club
 6         0x06     ACK acknowledge             ^ or spade
 7         0x07     BEL bell/beep               centre dot
 8         0x08     BS backspace           inverted text cursor
 9         0x09     HT horizontal tab           vert. oval
10         0x0A     LF line feed           |X> or inverted oval
11         0x0B     VT vertical tab         arrow or male symbol
12         0x0C     FF form feed           arrow or female symbol
13         0x0D     CR carriage return       single note symbol
14         0x0E     SO shift out              dual note symbol
15         0x0F     SI shift in                  sun symbol
16         0x10     DLE device link esc      down or right arrow
17         0x11     DC1 device control 1         left arrow
18         0x12     DC2 device control 2   tick or dual vert arrow
19         0x13     DC2 device control 3   diamond or dual exclamation
20         0x14     DC4 device control 4         pilcrow sign
21         0x15     NAK negative acknowl.       section symbol
22         0x16     SYN sync. idle              horizontal bar
23         0x17     ETB end trans. block    <X| or dual vert arrow underscored
24         0x18     CAN cancel                 left or up arrow
25         0x19     EM end medium              up or down arrow
26         0x1A     SUB substitute               right arrow
27         0x1B     ESC escape                   left arrow
28         0x1C     FS file separator              |____
29         0x1D     GS group separator       dual horizontal arrow
30         0x1E     RS record separator            up arrow
31         0x1F     US unit separator             down arrow
  • It seems very likely that maybe I have the wrong charset, though I'm not sure how to know whether I should use a different charset. – Jacob Jan 30 '16 at 21:17
  • Just try using ASCII instead of UTF-8, see what happens. Also try setting your TERM to vt100 and check what it does then. – Wyatt8740 Jan 31 '16 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.