2

Why are there still visual artifacts when terminal emulators draw text-based applications? This on recent computers that render 3D games and GUI windows including anti-aliased vector fonts with no artifacts.

I regularly see the following artifacts which reveal intermediate steps of the screen update process:

  • Terminal cursor motion (cursor blinks or jumps around the screen during update)
  • Tearing (part of screen shows old stuff while other part is showing new stuff)
  • Scrolling (scrolling is noticeable, instead of new scroll position being shown right away)

The artifacts are only seen for sub-second intervals, and not during most screen updates, but having grown up on flicker-free GUIs I would still like to know how to avoid them. All of the above artifacts (except scrolling) can be seen in e.g. the following ASCIInema video once it starts drawing the more complex screens: MapSCII - the whole world in your console!

I'm also specifically not talking about slow updates. It would be nice if updates were always instantaneous, but that's not always possible due to network and processing delays. What I mean here is that partially drawn screens are often visible for a brief moment. In most modern GUIs only fully finished screens are shown to the user, and artifacts of partial drawing are very rare.

It's my impression that the terminal emulation pipeline goes something like this:

  1. User presses a key on the keyboard
  2. Kernel passes keypress from keyboard driver to window system
  3. Window system passes keypress to terminal emulator
  4. Terminal emulator passes keypress to pseudo-terminal (pty) kernel device
  5. Pty interprets keypress and passes the result to text-based application
  6. Application performs command in response to keypress
  7. Application renders new screen (character cell grid) to internal buffer
  8. Application calls curses or other library to convert character cell grid to ANSI escape codes that will render an equivalent screen on the terminal
  9. Library writes those ANSI escape codes to the pty device
  10. Pty processes the written data somehow
  11. Terminal emulator reads processed data from pty in some chunks
  12. Terminal emulator calls window system to render the result of the ANSI escape codes in terminal window

Which of the above steps can slow down the process enough that the terminal emulator shows us intermediate rendering steps instead of showing only the final result?

  • It seems that the speed of hardware terminals (serial port connections) is dictated by their baud rate which can be changed with tcsetattr() but I read from multiple sources that the baud rate setting has no effect on the pseudo-terminal (pty) devices used by terminal emulators. Does this mean that Unix kernels do not deliberately rate-limit pty communications?

  • Do applications or rendering libraries (curses, etc.) send text and ANSI codes in multiple writes instead of trying to make do with only one write()?

  • Unix kernels have size limits on their internal I/O buffers, which affects things like the maximum amount of data that can be sent through a pipe without blocking. Does this affect rendering terminal screens with lots of detail (a screenful of text, lots of colors, etc.)? I imagine the combined text and ANSI escape codes could amount to so much data that it doesn't fit in the pty driver's buffer, which would split a screen update into several write operations by the application and several reads by the terminal emulator. If the terminal emulator were eager to display the results of each read before processing the next, this would cause the display to flicker until the final read in a batch has been processed.

  • Do terminal emulators or pty drivers have deliberate timeouts for batch processing so that their behavior more closely mimics hardware terminals, feels more natural, or addresses some other concern that was deemed more important than display speed?

Recently there has been some effort to make new terminal emulators that render faster (e.g. by pre-rendering fonts into OpenGL textures in video memory). But these efforts only seem to hasten the rendering of a character cell grid onto a screen bitmap once the grid has been calculated.

There seems to be something else going on that makes this stuff fundamentally slow even on a very fast computer. Think about it: if the terminal emulator processes all the ANSI codes to obtain a character cell grid before rendering anything into a screen bitmap, then it doesn't matter how slow the character-grid-to-bitmap rendering routines are - there should be no flicker (at least not the kind of flicker that clearly seems to correspond to cursor movement on a hardware terminal, which is what we often see). Even if the terminal emulator took a whole second to draw any given character cell grid on the screen, we would simply get a second of inactivity, not a second of flicker.

A similar issue is that the Unix clear and reset commands are incredibly slow for what they do (from a GUI user's perspective, they don't do anything more complex than redraw a bitmap). Perhaps for related reasons.

  • 1
    I don't see any slowness, even on truly remote xterm style windows. Are you using particularly old or slow hardware? – roaima Nov 23 '18 at 21:45
  • 1
    I find all the framebuffer consoles and terminal emulators that come bundled with "modern" desktop environments (gnome-terminal, etc) unbearably slow. I didn't even suffer to use them long enough to notice any flicker ;-) Try the vga text mode console or mlterm on a plain Xorg display. And also set xset r rate 200 200 for good measure ;-) – mosvy Nov 23 '18 at 23:47
  • 1
    @mosvy "Try the vga text mode console" – The vga text mode console is about 2 magnitudes slower than graphical emulators when processing large amount of data, I guess because it updates the contents inside the video card's area after every incoming chunk received. It becomes blazingly fast if you switch to another VT. – egmont Nov 24 '18 at 1:32
  • 2
    @egmont You're probably talking about the framebuffer console on linux, not about the vga 80x25 text mode. – mosvy Nov 24 '18 at 1:40
  • 1
    A related question is unix.stackexchange.com/questions/383218 . – JdeBP Nov 24 '18 at 12:40
2

I'd love to hear more details how exactly to trigger such prominent flickers, as I don't notice any while using my system.

On my system, VTE (the engine behind GNOME Terminal) can process about 10 MB/s of incoming data. The performance of other emulators aren't very far away from this either, maybe within a factor of 3 or 5 in both directions. This a lot, should be more than enough for flickes-less updates.

Keep in mind though that a fullscreen terminal might contain a few tens of thousands of character cells. UTF-8 characters consist of multiple bytes. Switching to different attributes (colors, boldness etc.) requires escape sequences that can go from 3–4 to easily 10–20 bytes (especially with 256-color and truecolor extensions). Truly complex layouts can thus require 100 kB or even larger amount of traffic. This sure cannot be passed over the tty line in a single step. I'm even uncertain whether certain apps (or screen drawing libraries) care to buffer the entire output in a single step. Perhaps they just use printf() and let stdio flush them after every 8 kB or so. This could be another reason for them being somewhat slow.

I'm not really familiar with the kernel's scheduling behavior, e.g. whether it needs to toggle back and forth between the two processes as well as user/kernel modes, or whether they can run simultaneously on a multi-threaded CPU. I truly hope though that they can run simultaneously on a multi-core CPU, which most CPUs are nowadays.

There is no intentional throttling in the story. There might be guesswork, though, when emulators decide whether to continue reading data, or update their screen. For example, if the terminal emulator processes the input faster than the app emits them, it sees it stalling after processing the first chunk, and thus might reasonably decide to update its UI.

The cursor is probably the most prominent to flicker, since the cursor moves along the screen as the contents are being updated. It cannot stay at the same place. If the emulator updates its screen just once while receiving the input data, and the cursor eventually remains at the same location, this flicker most likely become visible.

You might be interested in this proposal for atomic updates (discussions here) that would mostly solve this issue, if supported by both the terminal emulator as well as the application running inside.

You might also be interested in why the scrolling experience with the keyboard is necessarily jerky due to an interference between the keyboard repeat rate and the monitor refresh rate, something that isn't flickering per se, but causes an unpleasant experience.

  • 1
    Thanks for the very detailed answer :) I don't know why someone downvoted you. The proposal for atomic updates sounds like exactly what would solve the problem! It's awesome that people are working on it. "The goal of synchronized updates is to avoid showing a half-drawn screen, such as while paging through a document in an editor." "The following changes should be deferred until the end of a synchronized update: ... Changes to the cursor, such as blink, position, or style" – Lassi Nov 24 '18 at 9:31
  • 1
    I wouldn't say the flicker is "prominent" unless you are a stickler for this kind of stuff (as I am, after a decade of terminal use and being brought up on flicker-free GUIs). I'm trying to find some perfectly reproducible place where it can be seen. You can see the effect here asciinema.org/a/117813 after 00:15 (awesome that someone made such an application btw :D) – Lassi Nov 24 '18 at 9:38
  • 1
    The cursor flicker mentioned is already avoided in several full-screen TUI applications by simply hiding the cursor at the start of a redraw and showing it when finished. VIM and NeoVIM do this, for example. – JdeBP Nov 24 '18 at 12:34
  • @JdeBP That's great, that sounds like a fine fix for cursor motion. The atomic updates proposal would fix tearing/scrolling artifacts as well. It is put forth by the iTerm2 folks, which lends some hope that it may one day become widely adopted. – Lassi Nov 24 '18 at 12:56
  • 1
    @JdeBP "by simply hiding" – This prevents the cursor from showing up elsewhere, but it still potentially flickers at its desired place (assuming the cursor eventually goes back to its previous position after the repaint). – egmont Nov 24 '18 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.