Best practice for hiding virtual console while rendering video to framebuffer

Question

I'm currently rendering video in Linux directly to the framebuffer using GStreamer.

I was wondering how I would go about hiding the virtual console while rendering. I can stop the cursor from blinking, but that only works when no text changes on the console.

X seems to create a new screen accessible with Ctrl(+Alt)+F7 – is it possible to do something like that myself? Somehow be able to switch between a console & the rendering screen with Ctrl+Alt+F1 and Ctrl+Alt+F2.

Answer 1

X does not create a new screen.

To use the same display and input event devices that the kernel's built-in terminal emulator is using (to present its virtual terminals) a program must arrange to share them. The kernel's terminal emulator provides an API through which such a program can negotiate when it has responsibility for input and output, and when the kernel's built-in terminal emulator has.

This API is through ioctl() calls on a file descriptor that is open to a kernel virtual terminal character device. There are 64 of these devices in Linux, 16 in FreeBSD/PC-BSD. X does not create these. It opens an existing one — by convention one which no TUI programs are simultaneously trying to use as a kernel virtual terminal. In other words: By convention there's no TUI login session run on the kernel virtual terminal device that X opens and uses.

A program that shares with the kernel terminal emulator must …

• … tell the kernel terminal emulator to stop writing into the framebuffer to display output, or the cursor. This is done with the KDSETMODE ioctl() to set the nowadays quite misnamed KD_GRAPHICS mode. When in KD_TEXT mode the kernel terminal emulator doesn't nowadays usually have anything to do with display hardware being in an actual text mode. So-called framebuffer consoles have the display hardware in graphics mode. The distinction between KD_TEXT and KD_GRAPHICS modes is that in the former mode the kernel's terminal emulator will draw character glyphs onto the framebuffer as the terminal line discipline delivers output to it, and will also draw a cursor; whereas in the latter mode it will not do any drawing at all. These would actually be better thought of as "draw graphics" and "don't draw graphics" modes nowadays, were the wrong one not named "graphics". ☺
• … negotiate virtual terminal switching, if applicable. This is done with the VT_SETMODE ioctl(), with which the program can arrange to receive signals when the virtual terminal that it is using for the ioctl() calls is switched to or away from.
• … negotiate the handling of input with the kernel terminal emulator.
  • On Linux one might be reading from the input event subsystem directly, in which case the program tells the kernel's terminal emulator to stop reading those same input events, which it receives copies of, to stop translating them into characters, and to stop sending them off to the line discipline as input. How this is done varies:
    • The original way to do this was with the KDSKBMODE ioctl(), switching the virtual terminal into K_RAW mode. In this mode, the kernel terminal emulator still receives input events from the kernel's input event subsystem, but it performs no processing of them whatsoever, passing them to the line discipline as character input. However, this mechanism (which had its roots in the way that X worked before there was an input event subsystem) was broken, in that input was still being sent to the line discipline and still had to be drained. And it required that the termios input state for the terminal also be in raw mode, otherwise the raw scancodes would be misinterpreted as special characters such as the STOP or INTR characters by the line discipline.
    • A way, once considered to be better, to do this was with the KDSKBMODE ioctl(), switching the virtual terminal into K_OFF mode. In this mode, the kernel terminal emulator not only wouldn't process the input events, it wouldn't send them along to the line discipline. However, this mechanism was broken, because it was part of an K_OFF/K_RAW/K_CODE/K_XLATE mode switch. systemd and other similar systems would manage virtual terminal modes, and end up switching virtual terminals out of K_OFF mode.
    • The better way nowadays is to use the KDSKBMUTE flag. This turns off all input event processing without affecting, or being affected by, the K_RAW/K_CODE/K_XLATE mode switch.
  • On FreeBSD/PC-BSD, there's no separate input event character device in the first place. One reads keyboard input through the kernel virtual terminal anyway, so whilst one might want to switch it into scancode (K_RAW) or keycode (K_CODE) modes, one does not want to switch it off.

There are some interactions, here. An X server, for example, switches the virtual terminal into keycode mode, reads the keycodes and turns them into X keysyms, passing them through the X keyboard handling mechanisms. This means that the kernel's built-in terminal emulator never gets to perform the special processing for the Alt+Fn keyboard sequences. It is the X server that has to itself recognize Ctrl+Alt+Fn.

Further reading

• Arthur Taylor (2013-02-02). systemd should not call KDSKBMODE on a VT with X. systemd-devel.
• Adam Jackson (2012-11-16). [PATCH] vt: Drop K_OFF for VC_MUTE. Linux kernel mailing list.
• Adam Jackson (2012-11-16). [PATCH] linux: Prefer ioctl(KDSKBMUTE, 1) over ioctl(KDSKBMODE, K_OFF). xorg-devel.
• Michael K. Johnson (1994-06-01). Linux Programming Hints. Linux Journal.