46

I'm basically trying to figure out how one would go about making a GUI from absolute scratch with nothing but the linux kernel and programming in C.

I am not looking to create a GUI desktop environment from scratch, but I would like to create some desktop applications and in my search for knowledge, all the information I have been able to find is on GUI APIs and toolkits. I would like to know, at the very least for my understanding of the fundamentals of how linux GUI is made, how one would go about making a GUI environment or a GUI appllication without using any APIs or toolkits.

I am wondering if for example:

  1. existing APIs and toolkits work via system calls to the kernel (and the kernel is responsible at the lowest level for constructing a GUI image in pixels or something)

  2. these toolkits perform syscalls which simply pass information to screen drivers (is there a standard format for sending this information that all screen drivers abide by or do GUI APIs need to be able to output this information in multiple formats depending on the specific screen/driver?) and also if this is roughly true, does the the raw linux kernel usually just send information to the screen in the form of 8-bit characters?

I just really want to understand what happens between the linux kernel, and what I see on my screen (control/information flow through both software and hardware if you know, what format the information takes, etc). I would so greatly appreciate a detailed explanation, I understand this might be a dousie to explain in sufficient detail, but I think such an explanation would be a great resource for others who are curious and learning. For context I'm a 3rd year comp sci student who recently started programming in C for my Systems Programming course and I have an intermediate(or so I would describe it) understanding of linux and programming. Again Thank you to anyone who helps me out!!!

  • 9
    Note that a LOT of the work is done outside of the kernel. If you want to replace all that, you'll essentially be writing drivers, memory managers, etc. I suspect you'd want to include at least glibc, libmesa, and Xorg (or Wayland) in your dependencies. And honestly, if you're just starting, start by using a graphical toolkit like Gtk or Qt before doing all that yourself ;) – marcelm Feb 20 at 16:20
  • 3
    You can't avoid using any APIs. At a simplest level, you'd have to use the VGA BIOS APIs. – chrylis -cautiouslyoptimistic- Feb 20 at 17:52
  • 4
    The way the question is currently worded, there is room for an Android answer. – ctrl-alt-delor Feb 20 at 19:10
  • 4
    Can I ask how you came up with that username, considering how mature you seem in your writing – Bob Kerman Feb 21 at 2:05
  • 2
    @BobKerman I try to be as juxtaposed as possible. Keeps people guessing. – poopoopeepee123 Feb 21 at 10:04
62

How it works (Gnu/Linux + X11)

Overview

It looks something like this (not draws to scale)

┌───────────────────────────────────────────────┐
│                       User                    │
│     ┌─────────────────────────────────────────┤
│     │             Application                 │
│     │            ┌──────────┬─────┬─────┬─────┤
│     │            │      ... │ SDL │ GTK │ QT  │
│     │            ├──────────┴─────┴─────┴─────┤
│     │            │            xLib            │
│     │            ├────────────────────────────┤
├─────┴───┬────────┴──┐         X11             │
│   Gnu   │ Libraries │        Server           │
│   Tools │           │                         │
├─────────┘           │                         │ 
├─────────────────────┤                         │
│   Linux (kernel)    │                         │
├─────────────────────┴─────────────────────────┤
│                    Hardware                   │
└───────────────────────────────────────────────┘

We see from the diagram that X11 talks mostly with the hardware. However it needs to talk via the kernel, to initially get access to this hardware.

I am a bit hazy on the detail (and I think it changed since I last looked into it). There is a device /dev/mem that gives access to the whole of memory (I think physical memory), as most of the graphics hardware is memory mapped, this file (see everything is a file) can be used to access it. X11 would open the file (kernel uses file permissions to see if it can do this), then X11 uses mmap to map the file into virtual memory (make it look like memory), now the memory looks like memory. After mmap, the kernel is not involved.

X11 needs to know about the various graphics hardware, as it accesses it directly, via memory.

(this may have changes, specifically the security model, may no longer give access to ALL of the memory.)

Linux

At the bottom is Linux (the kernel): a small part of the system. It provides access to hardware, and implements security.

Gnu

Then Gnu (Libraries; bash; tools:ls, etc; C compiler, etc). Most of the operating system.

X11 server (e.g. x.org)

Then X11 (Or Wayland, or ...), the base GUI subsystem. This runs in user-land (outside of the kernel): it is just another process, with some privileges. The kernel does not get involved, except to give access to the hardware. And providing inter-process communication, so that other processes can talk with the X11 server.

X11 library

A simple abstraction to allow you to write code for X11.

GUI libraries

Libraries such as qt, gtk, sdl, are next — they make it easier to use X11, and work on other systems such as wayland, Microsoft's Windows, or MacOS.

Applications

Applications sit on top of the libraries.

Some low-level entry points, for programming

xlib

Using xlib, is a good way to learn about X11. However do some reading about X11 first.

SDL

SDL will give you low level access, direct to bit-planes for you to directly draw to.

Going lower

If you want to go lower, then I am not sure what good current options are, but here are some ideas.

Links

X11

https://en.wikipedia.org/wiki/X_Window_System

Modern ways

Writing this got my interest, so I had a look at what the modern fast way to do it is. Here are some links:

https://blogs.igalia.com/itoral/2014/07/29/a-brief-introduction-to-the-linux-graphics-stack/

| improve this answer | |
  • 16
    Great answer, but I do think there's something important missing: DRI, the Direct Rendering Infrastructure. With this, applications address hardware almost directly with a minimum of supervision by Xorg and the kernel (much like how Xorg mostly addresses hardware itself with a minimum of supervision of the kernel). This is used extensively to do hardware accelerated OpenGL though libmesa, for example. Then of course, nowadays there's Wayland which replaces Xorg, but of course at slightly different abstraction locations. It's a complex landscape :) – marcelm Feb 20 at 16:16
  • 4
    As marcelm says, hardware access nowadays goes through DRI (or DRM). /dev/mem provides access to very little memory nowadays. There’s also all the input stack, which is hidden behind libinput and libevdev now. – Stephen Kitt Feb 20 at 16:32
  • 10
    X11 is a protocol: it does not talking to hardware, it's formatted data sent between a client (the "application") and a server (probably x.org). The X11 server itself doesn't talk to hardware for output, either, and uses various file I/O and ioctl interfaces to the kernel to talk to input. Output is generally done through other kernel APIs like DRM to get frame buffers to fill and to talk to specialized drawing hardware like the GPU. Basically, constructing a GUI means sending and receiving data through kernel APIs. – Stephen M. Webb Feb 20 at 21:43
  • 2
    There is one extra layer that is glossed over here between X11 and the hardware - the device driver. But since the device driver is a kernel module you can be forgiven by just saying it is part of the kernel – slebetman Feb 21 at 3:19
  • 4
    @poopoopeepee123 I'd like to note that X11 itself (the API part) is just a protocol that runs over TCP/IP (on your machine your apps connect to localhost). This makes X11 very flexible by being able to run a GUI program on a different computer and displaying it on a different computer even in a different country. This also makes it possible to talk to X11 directly using standard network programming without using xlib if you study the protocol (it is also how people write things like clipboard managers by intercepting X11 events) – slebetman Feb 21 at 3:22
30

ctrl-alt-delor's answer gives you a good overview of the general architecture. For a more hands-on approach, I give you an answer regarding "nothing but the linux kernel and programming in C".

I like writing to the frame-buffer directly every now and then. The frame-buffer device driver will do all the tedious close-to-the-hardware "how will this eventually end up on a screen" stuff for you. You can do so right away with a root shell:

echo -n -e '\x00\x00\xFF' > /dev/fb0

It sets the very first (top left) pixel to red on my 32 bit framebuffer:

Screenshot of the framebuffer with the top left pixel red

You can totally do so from within C by opening /dev/fb0 and write bytes. Memory mapping can become your friend. This does only work without an X server or in a virtual console. Press Ctrl+Alt+F1 to access it.

PS: Visualising random data like your mouse movement can also be fun:

cat /dev/input/mouse0 > /dev/fb0

PPS: Please also note that virtually any real-world desktop application wants more direct access to the hardware for some fancy stuff like hardware acceleration for drawing, 3D and video rendering. The simple frame-buffer device won't do any of this well.

| improve this answer | |
  • Thanks for the answer! This combined with ctrl-alt-delor's answer provides me pretty much all the explanation I was looking for :) I'm new to actually posting questions on Stack Exchange and I wish I could give you both the checkmark! – poopoopeepee123 Feb 20 at 23:08
  • I think (but haven't checked) that this will also work with Xorg when it's using its fbdev driver. – Ruslan Feb 21 at 9:01
  • 5
    Mouse to frame buffer was the most fun I had all day – WhiteHotLoveTiger Feb 21 at 20:45
  • Also it's interesting to visualise things like your disk: cat /dev/sda1 > /dev/fb0 makes some cool patterns – Jachdich Feb 22 at 14:56
6

I would strongly recommend starting with ncurses.

Unlike more complex graphical systems, it is based purely on text, so there is no need to get bogged down in the details of screen drivers and graphics libraries. However the basic principles of putting windows on a screen, moving focus between windows, and so on, still hold true. And you can still do some drawing, at the level of single character blocks and ASCII art.

Of course you're still building this on top of a library, but it's a library which you can easily understand. And more than that, it's a library where the source code is freely available, fairly well documented, and not too impenetrable if you want to read it. You can even modify it yourself if you want to. Or you could look at all the library functions in there to find what the API needs to be, and write it yourself from scratch based on that design.

| improve this answer | |
  • 8
    "I would strongly recommend starting with nurses" Best advice so far. – Uncle Billy Feb 21 at 3:38
  • 7
    You probably mean ncurses which is not a GUI but a textual user interface. – Basile Starynkevitch Feb 21 at 6:03
  • 1
    @BasileStarynkevitch Oops - auto correct strikes again! :) Of course it's text-based, but it's still a GUI because it isn't a command line interface. Its basic drawing, layout and windowing capabilities translate directly into how a full-on window manager works, so it's a good place to start. – Graham Feb 21 at 7:30
  • 1
    @UncleBilly The last nurse I saw said "you'll just feel a prick". Thanks to auto-correct, that's true here too. ;) – Graham Feb 21 at 7:37
2

SunOS 5 had the DGA library, which provided device independent access to the different cg[3,6,14], TCX or LEO graphics adapters, which also was the thing which supported DOOM on the SPARC machines.

cg6 was a 8 bit, usually used in X11 as a pseudocolor visual but it could also provide an 8 bit truecolor while the tcx and leo is a 24 bit accelerated 3d display frame buffer (pseudocolor = a byte in videoram is an index into a large table which gives an 3x8 RGB value, the table's content can be changed easily.) The cg3 had about the same ability but it wasn't accelerated (the cg6 designers started afterwards another firm ... nVidia.)

The later devices like the PGX which was based on ATI Rage Pro chipset couldn't support truecolor and pseudocolor at the same time, which the earlier ones did. This forced a user to choose between old applications written for the pseudocolor model (or upgrade the sw if possible) and running only truecolor oriented apps.

Pseudocolor existed basically because what was videoram was awfully expensive in the mid 80s until 1992 or so. A color display which supported an usable workstation type resolution was also rather expensive (the 1984 Sun 2 black and white had a resolution of 1152x864 while a 1989 or so MG1 had 1600x1280 but b&w.)

I write this because i want to show the different requirements which X11 had to support.

| improve this answer | |
  • 7
    And none of this answers the question. Remember that there's a big difference between Stack Exchange and "normal" forums. – pipe Feb 20 at 21:45
  • Also the rather strange resolution 1152x864 is to be just below a megabyte. – Thorbjørn Ravn Andersen Feb 21 at 15:46
  • 4
    How is 1152x864 a strange resolution? Both numbers are divisible by 32, and it is exactly a 4x3 ratio. I used to run this res on a 17" monitor all the time. – Glen Yates Feb 21 at 18:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.