How to make a comprehensive set of possibilities for defining GNU-screen "command characters"?

Question

I want to somehow generate a comprehensive "menu" of all possible ways to set up the GNU-screen "command characters" for a standard, off-the-shelf "US keyboard" and a specific terminal emulator, say xfce4-terminal.

I imagine that this universe of possibilities would decompose into three lists:

• list A: all the possible values of the escape directive
• list B: all the possible values for the <CODE> parameter in the expression bindkey -k <CODE> command, together with the "null setting" (i.e. the .screenrc config in which no bindkey -k <CODE> command directive is used)
• list C: a mapping from any pair (a, b) (where a ∈ A, and b ∈ B) to an explicit description of how one would type the corresponding GNU-screen command character on a standard US keyboard, and assuming a specific terminal emulator (IOW, something equivalent to, for instance, "simultaneously press Ctrl and \").

Keep in mind, however, that these specs are a "best effort" by someone (me) who really does not understand the underlying basics. I hope that those who do understand these basics will be able to "read between the lines", and modify these specs as needed, while still retaining the spirit of the question (see Background), to render the question tractable.

I realize that the "menu" I'm hoping for may be very large, but I figure that it can't be intractably so, since, after all, the number of keys on a standard US keyboard is finite and not ginormous, and the set of fingers one can use for this purpose is even more so. (In case it matters, I could further stipulate that I am only interested in key combinations consisting of at most, say, 2 sequential "key chords", with at most 3 keys per key chord. By "key chord" I mean "set of keys to be pressed simultaneously".)

---

Background

(aka tl;dr)

This question is actually a follow-up to a comment by Gilles in a thread I started earlier. It turns out that most of what that comment says is beyond my grasp. I figure that there are some huge gaps in my understanding of the basics here, so big in fact that I cannot even articulate sufficiently clear questions to fill them.

In a nutshell, it is a huge mystery to me that, for example, the key combination Ctrl + \ can be used to type the "command character" for GNU-screen, whereas other similar-looking key combinations, like (maybe) Ctrl + ', cannot ¹.

Without a clear understanding of the underlying basics from the user (like me), the search for a suitable GNU-screen command character reduces to a sequence of isolated proposals ("How about Ctrl + H? After all, no one uses that for anything else.") that the user evaluates in turn until an acceptable one pops up.

How long this sequence of proposals and evaluations needs to go on depends on the size of the set of acceptable key combinations for that user. Clearly, this size will vary from one user to the next. In my case, it appears to be smaller than average, and as a result this approach has not yet given me an acceptable "command character" for GNU-screen.

The approach, in any case, seems to me inherently inefficient. It make more sense to me to be able to choose the best option from an explicit "universe" (i.e. "exhaustive set") of possibilities. This is what I'm trying to get at here.

---

EDIT: OK, after some studying, I now have a clear picture of how one types the (1-byte) characters in the ASCII range between \0001 and \0177, inclusive. These include all the "true" "control characters".

Also, I figure that list A can be described as all possible pairs of integers between \0001 and \0377, although probably many of these pairs can be ruled out as completely impractical. (E.g., those in which the first element of the pair is common "printable character", such as "e" or "8").

I'm still trying to figure out the following:

• how to type the (1-byte) characters in the ASCII range \0200 to \0377, inclusive; I expect there will be some variation across terminals and terminal emulators on this, but I have no sense at the moment of how chaotic the variation is; is there a subset of these characters on which there is a substantial consensus? If so, I would love to know what these characters are (and how to type them)

• how to get useful values for list B; I realize that these values are termcap codes; my difficulty here is not having a way to identify those termcap codes that map neatly to a convenient key combination; e.g. I know that the code F2 maps to F12 (sic), but I imagine that most termcap codes don't have such a neat association with a single key.

• how to complete list C, even for a specific terminal emulator, and the "null setting" from list B.

---

^{1Please, don't attempt to explain this mystery to me: many very knowledgeable, and very patient, people have tried, and I still don't get it. The "knowledge gap" between those who understand what's going on and me is so great that their answers are invariably as baffling to me as what they aim to address. What I'm hoping to achieve with this post is precisely to work around this huge knowledge gap, by casting the question as a search for, in essence, an algorithm (to construct a prescribed finite set of possibilities) that could be implemented by even someone who does not understand the underlying basics.}

Answer 1

To understand the answer to this question, you need to have some understanding of how keyboard input is processed. I refer you to How do keyboard input and text output work? for background. In this answer, I'm going to explain the relevant part in a different way, but I'll assume some general familiarity given by my earlier answer.

My answer here concerns typical unix systems; non-unix systems may behave differently. I will make some simplifications here and there; the additional complications are not relevant to answering this question. (This answer is complicated enough as it is.)

Most communication and storage, including the communication between a terminal-based application and a terminal (hardware or software), takes the form of a stream of bytes. A byte is a unit of information that can take 256 different values; it can be subdivided in 8 bits. Bytes are represented by a number between 0 and 255.

In order to transmit information, the parties need to agree on a way to encode this information as bytes. There are multiple ways to encode streams of characters as streams of bytes, but all of them are based on ASCII in one way or another. ASCII defines a correspondence between 7-bit values ranging from 0 to 127 and a set of 128 characters; this leaves an unused bit in each byte. The 128 characters fall into two categories:

• 95 printable characters: letters (A–Z, lowercase and uppercase), digits (0–9), space, some punctuation;
• 33 control characters.

Control characters encode orders and ancillary information sent to or from a terminal such as “move the cursor to the next line”, “ring the bell”, “wait for me”, “goodbye”, etc. Historical terminals introduced a key labeled “Control” (or “Ctrl” for short), which allowed users to enter control characters. To keep the electronics of the terminal simple, pressing Ctrl together with a key would perform a simple bit mask in the byte value normally sent by the character. For example, A sends the character A represented by the byte 65 (1000001 in binary); Ctrl+A sends the character represented by the byte value 1 (0000001 in binary), which is known as the “control-A” character, often written ^A.

Most control characters correspond to an uppercase letter, with the bit pattern 10xxxxx, with bit 6 set to 0 instead of 1. That accounts for 26 of them. 6 more control characters correspond to the punctuation characters that also have a bit pattern of the form 10xxxxx: these are @[\]^_ (see the ASCII printable character chart. In addition to the range 0–31, the character 127 is also a control character; it's known as “control-?” (? is 0111111; control-? is 1111111).

Over the years, non-ASCII byte values have been assigned different meanings. The world is converging to Unicode as the set of all characters anybody might want. The Unix world (as well as the Internet) has mostly standardized on UTF-8 as a way to encode characters as byte sequences. UTF-8 maintains compatibility with ASCII by assigning the same character as ASCII to any byte in the range 0–127, and using sequences 2 to 4 bytes in the range 128–255 to represent the million or so other characters. Some other character encodings are used in the unix world; most are based on ASCII and have different meanings for bytes above 128.

I can now answer one of your follow-up questions:

how to type the (1-byte) characters in the ASCII range \0200 to \0377

That depends what character encoding you use. If you use UTF-8, the most common one, sending these individual bytes is impossible, because they're only used as part of sequences of 2 to 4 bytes that represent a single character.

As for

list A: all the possible values of the escape directive

that's just byte values. The escape directive requires a two-byte parameter. If Screen receives the first byte value from the terminal, it decides that its escape key has been pressed. If the next byte sent by the terminal is the second byte from the escape setting, then Screen decides that you wanted to send that first byte to the application running inside the Screen window after all.

The description of the bind command explains how to specify byte values in a way that Screen understands. Where the documentation reads “character”, read “byte” instead.

Before we get to list B, we need to understand keychords. A keychord is the press of a key together with modifiers such as Ctrl, Shift, etc. We saw earlier that all information transmitted by the terminal is encoded as a stream of bytes. To keep things simple, all printable characters are encoded in the standard way, as one byte in the range 32–126 if they're ASCII characters and as bytes in the range 128–255 for other characters. This leaves only control characters to encode function keys and characters with modifiers other than Shift. But there are only 33 control characters!

A few function keys send a control character. For example, the Tab key sends ^I (byte value 9), because byte 9 is the TAB control character that instructs a printer to move to the next tab column. The Return key sends ^M (byte value 13), because byte 13 is the CR control character that instructs a printer to move its head to the beginning of the line. For similar reasons, Escape sends ^[, and BackSpace sends either ^H or ^? due to some historical waffling that I won't discuss here.

Most function keys and keychords send an escape sequence: a sequence of bytes starting with byte 27, the escape character (ESC) defined by ASCII, which happens to be ^[ (control-[). Different terminals sent different escape sequences. There are standards, but they don't define encodings for all keychords, far from it, and to some extent there are competing standards.

We're now ready to understand

list B: all the possible values for the <CODE> parameter in the expression bindkey -k <CODE>

The Screen documentation explains that these codes are termcap keyboard capability names. Termcap is a programming library that applications can use to abstract themselves from the variations between terminals. (It has now been mostly supplanted by Terminfo.) The Termcap database specifically contains information about the terminal such as the number of lines and columns (back when Termcap appeared, terminals were hardware devices, for which the concept of resizing didn't apply), the byte sequences (often beginning with ESC) that an application could use to perform operations such as moving the cursor or clearing the screen, and the byte sequences send by various keys. The symbolic names given by Termcap to function keys are what you can use after bindkey -k.

The Termcap manual lists all the entries in that database. The entries all have two-character names; the FreeBSD manual also gives a slightly more expressive name for each entry. The entries where FreeBSD lists key_SOMETHING in the first columns are the ones that describe function keys; the <CODE> you need for bindkey -k is the name in the second column, such as kl for Left, k1 for F1, F1 for F11, etc.

You'll notice that this database is missing a lot of keychords. If there's no entry for a keychord in this database, then there's no name you can use for the key with bindkey -k. Note that the set of supported keys varies from unix variant to unix variant.

bindkey can also be passed an escape sequence. To use this feature, you need to know what your terminal sends for the keychord you're interested in. While different terminals send different escape sequences for the same keychord, going from escape sequence to keychord is, fortunately, rarely ambiguous: very few escape sequences correspond to different keychords on different terminals.

You can find out what escape sequence a keychord sends by pressing Ctrl+V then the keychord. In a terminal in its default mode, as well as on the command line of all common shells, Ctrl+V means “interpret the next byte literally”. If it's followed by an escape sequence, this causes the ESC byte to be inserted literally instead of initiating the parsing of an escape sequence. Since escape sequences almost always consist of printable characters after ESC, this effectively inserts the escape sequence literally. For example, press Ctrl+V then Ctrl+Left to see what escape sequence Ctrl+Left sends: you'll see something like ^[O5D where ^[ is a visual representation of the ESC control character. (Once again, your terminal may send a different escape sequence.)

As for the null setting, when Screen reads an ESC byte, it enters escape sequence parsing mode. Each new byte is added to the accumulated escape sequence. If the accumulated sequence has an associated binding, Screen exits escape sequence parsing mode and fires the binding. If the accumulated sequence is not the prefix of any sequence with an associated binding, Screen exits escape sequence parsing mode and the accumulated sequence is discarded. So the null setting here is a complicated form of “nothing happens”.

After all this work, let's turn to

list C: a mapping from any pair (a, b) (where a ∈ A, and b ∈ B) to an explicit description of how one would type the corresponding GNU-screen command character on a standard US keyboard, and assuming a specific terminal emulator

As I hinted above, specific terminal emulator is important here: different terminals encode keychords in different ways, and some terminals can be configured in different ways. The mapping doesn't correspond to a pair (a, b): A × B is not an interesting set. Most keychords are mapped to either a printable character (which, as we saw above, extends A) or an escape sequence (which, as we saw above, extends B). In other words, the mapping is to a superset of A ∪ B.

Unfortunately for you, many terminals don't fully document how to send escape sequences. Fortunately for you, this is rarely needed. Instead of working from escape sequence to keychord, work from keychord to escape sequence. This can be determined for each terminal using Ctrl+V as described above.

Some terminals, notably xterm, can be configured to encode keychords in a systematic way. See Problems with keybindings when using terminal for an Emacs-oriented discussion. Unfortunately, this does not include the vte library that many terminal emulators use, especially in the GNOME world.