The question is quite broad, and a lot more could be written on the subject than this answer covers. I have tried to provide a historical perspective on the evolution of Linux graphics. Graphics, windowing systems, and graphical user interfaces (GUIs) on Linux have gone through a lot of changes since the early 1990s, when the X Window System (X11) was ported to Linux.
The X Window System
The X Window System was developed at MIT in the 1980s. The name X11 refers to protocol version 11 of the X protocol, but X10 was also used outside of MIT before being replaced by version 11 in 1987.
The X Window System was designed to work on graphics systems that were the state of the art in the 1980s. A typical workstation had a single frame buffer connected to a simple CRT controller which displayed the contents of the frame buffer on a display monitor. Computing before the PC and workstation era was done via serial line ("dumb") terminals connected to central computers situated in computer machine rooms. This historical context influenced the design of X11: graphical applications could be run on remote computers with the user interacting with the program using terminals with graphics capabilities. The "terminal" could be a workstation or a dedicated X terminal.
X11 was designed as a server-client system. The X server was the only part communicating directly with the graphics hardware. The X clients are application programs talking to the server using the X protocol, either using a local Unix Domain socket or a TCP/IP connection. The X protocol is used by the client to both send requests to the server and receive event messages from the server.
Requests include messages for:
- window creation
- mapping/unmapping a window: making the window visible/invisible
- drawing on the window: draw pixels, lines, arcs, ovals, pixmaps, etc.
- displaying text using a specified font, size and style
- moving and resizing windows, changing the stacking order of windows, etc.
Clients receive messages (not an exhaustive list):
- replies to requests
- keypress and mouse click events
- expose events (an area of a window needs to be redrawn)
- focus gain/loss events
To enable the user to handle the windows on screen, for example, move, resize, close, raise and lower a window, a particular application called the window manager is provided. The window manager can also display window decorations like borders, title bars, and global menus.
You could say the X11 server is quite "high level", as it handles (or at least traditionally handled) all kinds of resources: windows, fonts, pixmaps, colormaps, graphic contexts (things like foreground/background color, line width, etc). In addition to this, the server takes care of things like window parent-child relationships and the stacking order of windows.
The X protocol is designed to be extensible. The X server can be taught to do new tricks, and new opecodes are added to the protocol to have the server perform those tricks. For example, the XRender extension introduces a way to handle transparency ("alpha blending"). This extension was introduced mainly to support anti-aliased fonts, but has also been used for desktop effects like drop shadows on windows. The RandR ("Resize and Rotate") extension makes it possible to resize, rotate and reflect the root window on the screen. This enables you to project the screen using a projector that is upside down, or to use a tilted monitor.
The GLX extension (OpenGL Extension to the X Window System) makes it possible to use OpenGL in a window provided by the X server. The calls to OpenGL are embedded in X protocol requests.
At some point in the evolution of X11, font handling was moved to be handled by the client. The reasons behind this change is discussed in New Evolutions in the X Window System.
In the early 2000s, display hardware had come a long way from the simple black-and-white bitmapped displays that existed when development of X started in the 1980s. The X11 relative overhead of the inter-process communication (IPC) model had grown too big, even when using a local socket. The solution to this was to abandon the principle that the X server is the only part that talks directly to the hardware, and let the clients talk to the graphics card directly. The Direct Rendering Infrastructure (DRI) was born.
DRI allows an X client app to bypass the X server and render directly on the graphics adapter. Because several direct rendering applications in addition to the traditional X server can be active at the same time, a kernel component called the Direct Rendering Manager was introduced to arbitrate access to hardware. There are three versions of the DRI architecture, the original DRI (obsolete), DRI2, and DRI3.
Compositing Window Managers
The next innovation to enter the Linux graphics scene was the compositing window manager. Traditionally, each X client application was responsible for repainting its windows (partially, or the whole window) on demand. The X server sent the application an Expose event when a repaint was needed as the result of the window being mapped on the screen, or if it is no longer obscured by some other window. When an overlapping window is removed, the window beneath it is exposed. Failing to repaint this area lead to the old contents still being displayed. https://en.wikipedia.org/wiki/Visual_artifact
A compositing window manager changes this. Applications render to their own off-screen buffers, each of which is kind of a separate screen with exclusive access by the application owning the buffer. It is the task of the compositing window manager to display these buffers in windows on a real screen, clipping any windows that are obscured by other windows or partially off-screen. The window manager displays a "composition" of the windows.
A compositing manager can typically also display animated effects, like scaling, warping, fading, rotating, blurring the windows. For example, moving a window can make it wobble, or virtual desktops can be displayed on the side of a rotating cube.
Kernel Mode Setting
The X server traditionally also took care of setting the modes of the graphics adapter, like resolution and refresh rates. The mode setting has since been moved to a Linux kernel component called Kernel Mode Setting (KMS). This solved a lot of problems with switching between Linux's virtual consoles.
The X server also had knowledge of the input devices and, for example, the type of mouse had to be specified in the X configuration. The X server has been relieved of this task with the introduction of the evdev subsystem of the Linux kernel, which provides a generic input event interface.
With all these developments, a lot of the tasks performed by the X server have moved outside of the X server. Using direct rendering, clients don't use the X protocol anymore. Thanks to KMS, the X server does not need to muck with the low level programming of graphics adapters. With evdev, input device handling was simplified in the X server. When using a compositing window manager rearranging and warping windows, the X server has no idea what's going on on the screen anymore. "The window manager is the new X server".
Wayland came about as a result of the realization that the X server process had little left to do, and by cutting out the middle-man (the X server), a much simpler desktop graphics system could be achieved. Backward compatibility is provided via Xwayland, a modified Xorg server that displays top level X windows using Wayland surfaces.
Strictly speaking, Wayland is just a protocol that defines how clients communicate with the display server. The Wayland protocol is quite unlike the X protocol: the Wayland protocol does not define messages to draw graphics or text, nor does it handle fonts.
In the Wayland architecture, the window manager and the display server are merged into one software component, the compositing window manager. Clients can request, via a software library using the Wayland protocol, a surface to draw on. A "surface is an object representing a rectangular area on the screen, defined by a location, size and pixel content".
Clients render into off-screen buffers, which are then attached to a surface, producing output on the screen. The client can use various APIs to do the rendering: OpenGL, OpenGL ES, etc. ("What is the drawing API? Whatever you want it to be") Double buffering is used: a client updates its image using a second buffer, and when that buffer contains a coherent image, it is switched to be displayed at the next display monitor vertical blanking interval. Wayland's motto is: "Every frame is perfect", i.e. windows do not tear, flicker or flash.
Input handling in Wayland goes through the compositor, which is the only component knowing which window is under the mouse cursor (remember, the compositor may also have warped the windows). The compositor transforms the screen coordinates to window-local coordinates of the appropriate window and sends the event to the client.
If you are interested in the story that led to the creation of Wayland, I recommend watching Daniel Stone's hilarious presentation The real story behind Wayland and X.