Why do we need so many terminal emulator packages and what is their use?

Question

This is something that has been so confusing to me for a long time.
I am using Linux but I have gaps in my understanding of certain aspects and one is about the terminal.
As I came to understand it what we mean with the console is an emulator of old terminals.
But I can not understand what is the deal with this.
As I read the best old terminals were the VT series and the vector graphics type (Tektronix).
So I assume that the current emulators emulate these.
Now my needs in Linux concerning display presentation has reached as far as using a colorscheme for vim and appreciating colors when doing diff in files.
But when I have issues I usually find advice to set xterm-256 or screen-256 etc.
Additionally when I do: ls /lib/terminfo/ I see about a dozen of directories with configuration for (I assume) different kind of emulator types.

So my question is, how does these xterm-256 or screen-256 and the rest fit in the idea of emulating just the top terminals in the past decades?
What is the need to have so many terminal types? Is it something I should look into more?
And why today with the modern technology need to emulate these old terminals like VT and not have something new?

Perhaps my needs are too simple too appreciate the subtleties of this but it is has been something that puzzled me a lot. Since for instance if I have an issue with a colorscheme I just copy paste what I find in google about TERM etc without really understanding what am I doing or what is the problem.

If someone helps me undertand this it would be really appreciated

Answer 1

Weird aspects of Unix usually exist for good reason, so you're right to look for one. In this case, though, the good reason has long since become obsolete, and you're looking at an antique artifact of a bygone era.

Just about the only "terminal" in existence today is xterm & variants. Their capabilities vary very slightly, in ways that matter to only a few programs. If you just use xterm, and never touch the TERM variable or peek at the terminfo database, your life will generally be better.

The TERM variable communicates information about the terminal to the application through the environment, cf. man xterm. Changing it doesn't change the terminal; it just represents different terminal functionality to the application.

In the days of hardwired terminals, it was necessary to set TERM to represent the attached terminal. In the case of xterm, the software can set the variable itself. A quick tour of the vim docs shows (as you mention in your comment) that you have to change it to support color. That's progress for you.

why today ... emulate these old terminals like VT and not have something new?

The answer is as much anthropological as technical.

Before the GUI, access to Unix machines was via dumb terminals e.g. VT-100. Shells and utilities like top already existed. When the GUI became technologically practical (in which X played a role) in the 1980s, Unix users still needed to use those programs, so xterm was invented to emulate ye olde VT-100.

It was meant as a stopgap. "Everyone knew" that terminals were the past and GUIs were the future, and everyone expected "everything" to be accessed via the GUI. The original Macintosh, for example, had no arrow keys because why would you need them? Surely the cryptic Unix command line, with its missing vowels and helpless help

$ help
help: not found

would soon go the way of drum memory and punch cards. And that did come to pass, in a way: 9 users in 10 running Windows or OS X never see the command line except when tech support drops by to fix something.

Then two things happened to the Unix GUI, such as it was. Windows in particular drained the money out of the market. There was a big move to standardize it (cf. Sun News and OSF Motif), and then it ground to a halt around 1990. Just about that time the Internet took off, and things graphical in Unix moved into the web browser. The motivation and the money (pretty much the same thing) to engineer a complete GUI for Unix and render everything in section 8 of the manual obsolete disappeared.

There is another reason, too, that very few foresaw: the command line has certain advantages over the GUI. Pipelines and regular expressions are remarkably powerful, not to mention repeatable with shell history and scripts. Even in the context of a GUI, the command line remained useful. So much so that it continues to be enhanced even today.

As your question suggests, what's needed is a re-examination of the assumption that the GUI would triumph, and a re-invention of the terminal as an integral part of it. We need a new terminal, with proportional fonts and bit-addressable graphics in the terminal.

Unfortunately, no one seems ready to do that. No corporate entity will undertake it; the market is huge, but still only a tiny proportion of computer users. The logical funder would be a government agency like DARPA, but human-interface research is considered "done" these days (didn't we already invent the GUI?). Until more people -- lots more people -- recognize the need, xterm is your friend, and likely to be your grandson's friend, too.