110

There are special files in Linux that are not really files.

The most notable and clear examples of these are in the dev folder, "files" like:

  • /dev/null - Ignores anything you write to the file
  • /dev/random - Outputs random data instead of the contents of a file
  • /dev/tcp - Sends any data you write to this file over the network

First of all, what is the name of these types of "files" that are really some sort of script or binary in disguise?

Second, how are they created? Are these files built into the system at a kernel level, or is there a way to create a "magic file" yourself (how about a /dev/rickroll)?

  • 1
    I had no clue how to tag this question, especially since I don't know the name of what I'm looking for. Feel free to edit in any relevant tags. – IQAndreas Nov 6 '15 at 6:42
  • 15
    BTW, this is a fundamental part of the design of unix and unix-like operating systems: (almost) everything is a file, or can be made to look like a file. – cas Nov 6 '15 at 7:48
  • 5
    Also See: mknod(2) man 2 mknod – RobertL Nov 6 '15 at 7:54
  • 4
    These are "device nodes". However the ones you mentioned - unlike those associated with disks, keyboard, mice, audio-cards, and other devices - are so-called "pseudo-devices", as they're not "real" devices and only exists in the kernel. It's possible to create new ones, by writing a suitable device-driver and add it to the kernel (eg. a pseudo-device for monitoring some activity on the computer). Before the /dev-directory existed on disk - these days it's a virtual file-system (of type devfs) created by the kernel. – Baard Kopperud Nov 6 '15 at 15:18
  • 10
    All files, even "real" files, are software artifacts. The software behind every device, file, socket, special file, or something yet to be invented provides a table of functions to handle open(), read(), close(), etc. After that, it's up to the software – waltinator Nov 6 '15 at 21:25
100

/dev/zero is an example of a "special file" — particularly, a "device node". Normally these get created by the distro installation process, but you can totally create them yourself if you want to.

If you ask ls about /dev/zero:

# ls -l /dev/zero
crw-rw-rw- 1 root root 1, 5  Nov 5 09:34 /dev/zero

The "c" at the start tells you that this is a "character device"; the other type is "block device" (printed by ls as "b"). Very roughly, random-access devices like harddisks tend to be block devices, while sequential things like tape drives or your sound card tend to be character devices.

The "1, 5" part is the "major device number" and the "minor device number".

With this information, we can use the mknod command to make our very own device node:

# mknod foobar c 1 5

This creates a new file named foobar, in the current folder, which does exactly the same thing as /dev/zero. (You can of course set different permissions on it if you want.) All this "file" really contains is the three items above — device type, major number, minor number. You can use ls to look up the codes for other devices and recreate those too. When you get bored, just use rm to remove the device nodes you just created.

Basically the major number tells the Linux kernel which device driver to talk to, and the minor number tells the device driver which device you're talking about. (E.g., you probably have one SATA controller, but maybe multiple harddisks plugged into it.)

If you want to invent new devices that do something new... well, you'll need to edit the source code for the Linux kernel and compile your own custom kernel. So let's not do that! :-) But you can add device files that duplicate the ones you've already got just fine. An automated system like udev is basically just watching for device events and calling mknod / rm for you automatically. Nothing more magic than that.

There are still other kinds of special files:

  • Linux considers a directory to be a special kind of file. (Usually you can't directly open a directory, but if you could, you'd find it's a normal file that contains data in a special format, and tells the kernel where to find all the files in that directory.)

  • A symlink is a special file. (But a hard link isn't.) You can create symlinks using the ln -s command. (Look up the manpage for it.)

  • There's also a thing called a "named pipe" or "FIFO" (first-in, first-out queue). You can create one with mkfifo. A FIFO is a magical file that can be opened by two programs at once — one reading, one writing. When this happens, it works like a normal shell pipe. But you can start each program separately...

A file that isn't "special" in any way is called a "regular file". You will occasionally see mention of this in Unix documentation. That's what it means; a file that isn't a device node or a symlink or whatever. Just a normal, every day file with no magical properties.

  • 4
    There's also one more type of special file, a Unix domain socket bound to the filesystem. – Brian Bi Nov 6 '15 at 18:50
  • 8
    If you want to play with mknod, run cat /proc/devices to see the major numbers for all drivers. Which brings us to yet another kind of special file the /proc file system (this answer talks about it). – ugoren Nov 6 '15 at 21:57
  • 8
    Other Unices have invented their own special files, e.g. Solaris had doors. – Kevin Nov 6 '15 at 22:03
  • 6
    Minor nitpick: you don't have to recompile the kernel to write a new character/block device :) crashcourse.ca/introduction-linux-kernel-programming/… Otherwise this is a really good answer, +1! – Commander Coriander Salamander Nov 7 '15 at 19:50
  • 1
    @MathematicalOrchid: A step your reply is missing (or at least only stating implicitly) is the fact that those special files are not disguised shell scripts or binaries at all (as the question implied), but rather an interface to access functionality that is present in the OS kernel. – Dreamer Nov 9 '15 at 22:15
34

Most of the /dev entries are block device inodes or character device inodes. Wikipedia has many detailsabout that, which I am not going to repeat.

But /dev/tcp which is mentioned in your question is not explained by any of the existing answers. /dev/tcp and /dev/udp are different from most other /dev entries. The block and character devices are implemented by the kernel, but /dev/tcp and /dev/udp are implemented in user mode.

The bash shell is one program which has an implementation of /dev/tcp and /dev/udp (copied from ksh93). When you try to open a path beneath those with bash redirection operators, it will not perform an ordinary open system call. Instead bash will create a TCP socket and connect it to the specified port.

That is implemented in user mode and only in some programs as can be seen in the following example which demonstrates the difference between letting bash and cat try to open /dev/tcp/::1/22

$ cat /dev/tcp/::1/22
cat: /dev/tcp/::1/22: No such file or directory
$ cat < /dev/tcp/::1/22
SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.3

A difference with ksh93 is that bash will only do those TCP connections with redirection operators, not in the other places where it may open files like the source or . builtin.

  • Also, GNU awk gawk similarly special-cases /inet{,4,6}/{tcp,udp}/$port/$remote/$rport, since somewhere around 2010 (I don't recall exactly and can't find release notes). – dave_thompson_085 Nov 6 '15 at 23:28
  • 6
    IMO, a better way to state the point about /dev/tcp is that it's NOT a file. There is never a file called this. Bash's syntax for opening sockets uses the string /dev/tcp/address like a filename, but calling it a "file implemented in user space" just sounds weird. Interesting that ksh hooks those filenames for everything, not just redirects, though. That's closer to "implementing a file". – Peter Cordes Nov 7 '15 at 1:52
  • @PeterCordes i believe UWIN sets these up as actual files. and i think 3dfs does the same. remember, bash only copied this behavior, but it originates elsewhere. – mikeserv Nov 7 '15 at 3:37
19

In addition of device nodes explained in other answers (created with mknod(2) or supplied by some devfs), Linux has other "magical" files provided by special virtual file systems, in particular in /proc/ (see proc(5), read about procfs) and in /sys/ (read about sysfs).

These pseudo files (which appear -e.g. to stat(2)- as ordinary files, not as devices) are a virtual view provided by the kernel; in particular, reading from /proc/ (e.g. with cat /proc/$$/maps, or by open(2)-ing /proc/self/status in your program) generally does not involve any physical I/O from disk or network, so is quite fast.

To create some additional pseudo-file in /proc/ you generally should write your own kernel module and load it (see e.g. this).

  • 3
    AFAIK the information on extending /proc is outdated. While still technically possible, /proc (or rather procfs) should only hold information on running processes. All other pseudo-files including those that contain runtime information or configuration options for the kernel should go into /sys (sysfs). There are still some non process-related pseudo files in /proc (e.g. meminfo, cpuinfo) for compatibility reasons, but new pseudo files should go into sysfs. – Dreamer Nov 9 '15 at 22:22
13

They're called device nodes, and are created either manually with mknod or automatically by udev. They are typically file-like interfaces to character or block devices with drivers in the kernel - e.g. disks are block devices, ttys and serial ports etc are character devices.

There are other "special" file types too, including named pipes and fifos and sockets.

9

As other users have already explained in great detail, special files require code to back them up. However, nobody seems to have mentioned that Linux provides several ways to write that code in userspace:

A. FUSE (Filesystem in USErspace) allows you to write something like /proc without risk of crashing the kernel and do it in a language/runtime of your choice, such as Go, Node.js, Perl, PHP, Python, Ruby, Rust, etc..

It also has the advantage that FUSE filesystems can be mounted without sudo because they run as the user doing the mounting.

Here are some examples of things people have written using FUSE:

  • mp3fs (View your FLAC files as MP3 files that get created on-the-fly when you copy/click-drag them to your MP3 player)
  • PyTagsFS (View your media in a tree of virtual folders built from the metadata tags)
  • fuse-zip (Mount Zip files as folders)
  • FuseISO (Mount ISOs without root permissions)
  • iFUSE (Mount iDevices)
  • FuseDAV (Mount WebDAV shares)
  • fuse-exfat (Mount exFAT-formatted filesystems)
  • ntfs-3g (The Linux NTFS driver)

B. If you want to create a virtual input device like a keyboard, mouse, joystick, etc. (eg. to write a userspace driver for a USB device you're talking to using libusb), there's uinput.

Bindings for it are harder to find, but I know they exist for Go (Keyboard-only), Python, and Ruby (2).

Examples of real-world uinput use include:

  • G15Daemon (Linux driver for the LCD and gaming keys on Logitech G15 gaming keyboards)
  • ds4drv (Driver for Sony DualShock 4 controllers)
  • xboxdrv (Alternative XBox 360 controller driver and Linux equivalent to x360ce so badly designed games like Runner2: Future Legend of Rhythm Alien can think they're talking to a real XBox controller when they're not)
  • The old Wiimote drivers like cwiid that were required before someone finally wrote a kernel Wiimote driver so support would be available by default.

C. For generic character devices, there's CUSE (Character devices in USErspace). It's much less popular though.

The only user of the CUSE API that I'm personally aware of is the same program which prompted its creation: osspd, which implements /dev/dsp, /dev/adsp, and /dev/mixer (the OSS audio API) in userspace so they can be routed through PulseAudio or dmix.

The only CUSE binding I was able to find is cusepy, which hasn't been updated since 2010.

D. You may not need a new special file at all.

For example, you can open up raw communication with any USB device using libusb (List of bindings on the page) and then communicate with other programs through some other mechanism (TCP/UDP sockets, reading/writing stdin/stdout or regular files on disk, etc.).

  • 1
    cusepy might not have been updated in a while (in fact, it's never been updated; it only has one commit!), but having just written a character device using cusepy a few weeks ago, I can confirm it still works fine. It was missing a few functions related to implementing poll, but since cusepy uses ctypes and the bindings are autogenerated based on C header files, fixing any missing functions is just a matter of adding the desired function name to the list of exported functions in the setup.py. – Aleksi Torhamo Nov 8 '15 at 22:31
  • 1
    Another interesting example of FUSE usage is sshfs. It allows you to browse remote file system as if it was local using SSH connection underneath. – Mr. Deathless Nov 13 '15 at 13:23
  • @Mr.Deathless Yeah. I actually use that and meant to mention it but I forgot. – ssokolow Nov 13 '15 at 18:54
6

The book Linux Device Drivers (highly recommended) explains this in detail, and even has you create a kernel module that does this as an example, but in a nutshell, each device driver has specific functions that get called when a file is opened, closed, read, written, etc. The "special" files just do something special inside those functions, instead of accessing the storage hardware on a disk.

For example, the write function for /dev/null just does nothing, ignoring the bytes. The read function for /dev/random returns a random number.

1

mount -t devtmpfs

It is also interesting to see that in modern systems, /dev is normally filesystem type that can be mounted wherever you want. Ubuntu 16.04:

mkdir d
sudo mount -t devtmpfs none d
head -c 10 d/random
sudo umount d

This is enabled by CONFIG_DEVTMPFS=y, and allows the kernel itself to create and destroy device files as needed.

CONFIG_DEVTMPFS_MOUNT=y

This option makes the kernel auto-mount devtmpfs on /dev.

drivers/base/Kconfig documents:

config DEVTMPFS_MOUNT
    bool "Automount devtmpfs at /dev, after the kernel mounted the rootfs"
    depends on DEVTMPFS
    help
      This will instruct the kernel to automatically mount the
      devtmpfs filesystem at /dev, directly after the kernel has
      mounted the root filesystem. The behavior can be overridden
      with the commandline parameter: devtmpfs.mount=0|1.
      This option does not affect initramfs based booting, here
      the devtmpfs filesystem always needs to be mounted manually
      after the rootfs is mounted.
      With this option enabled, it allows to bring up a system in
      rescue mode with init=/bin/sh, even when the /dev directory
      on the rootfs is completely empty.

file_operations

Finally, you should create your own character device kernel module to see exactly what is going on.

Here is a minimal runnable example: Understanding character device (or character special) files

The most important step, is setting up the file_operations struct, e.g.:

static const struct file_operations fops = {
    .owner = THIS_MODULE,
    .read = read,
    .open = open,
};

static int myinit(void)
{
    major = register_chrdev(0, NAME, &fops);
    return 0;
}

which contains function pointers that get called for each file-related system call.

It then becomes obvious that you override those file-related system calls to do whatever you want, and so this is how the kernel implements devices like /dev/zero.

Create /dev entries at automatically without mknod

The final mystery is how the kernel automatically creates /dev entries.

The mechanism can be observed by making a kernel module that does that yourself as shown at: https://stackoverflow.com/questions/5970595/how-to-create-a-device-node-from-the-init-module-code-of-a-linux-kernel-module/45531867#45531867 and comes down to a device_create call.

  • In OpenBSD there's a script MAKEDEV that simplifies this a little, see man.openbsd.org/MAKEDEV.8 Not sure why Linux doesn't have it except it's way more complicated. Maybe parts could be adapted. You can say MKNOD tty for example and it handles the details. – Alan Corey Feb 22 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.