16

I’m asking because string comparisons are slow, but indexing is fast, and a lot of scripts I write are in bash, which to my knowledge performs a full string lookup for every executable call. All those ls’s and grep’s would be a little bit faster without performing a string lookup on each step. Of course, this now delves into compiler optimization.

Anyways, is there a way to directly invoke a program in Linux using only its inode number (assuming you only had to look it up once for all invocations)?

14
  • 5
    What is this "full string lookup" you keep referencing? Nov 19, 2022 at 22:27
  • 16
    If you really are using ls in a script, that’s a sign you’re doing something wrong, and you’d probably benefit more by avoiding the ls invocation altogether than by optimising it. Nov 19, 2022 at 23:14
  • 26
    The name lookups the kernel needs to make are likely not all that significant compared to all the other bookkeeping involved in starting a new process and executing a new program in it. If it's ls and grep you're thinking, you might get better results by switching to a programming language with better tooling for data processing
    – ilkkachu
    Nov 20, 2022 at 9:53
  • 8
  • 17
    There seems to be an implicit assumption here that the shell is doing a new PATH lookup on each execution of a binary. That's not true; there's a cache in bash preventing new PATH lookups in userspace, and the kernel has its own aggressive caching on its side of the userspace/kernel divide. The fork()+execve() part of running a new executable is much, much slower. Nov 20, 2022 at 17:53

4 Answers 4

41

The short answer is no.

The longer answer is that linux user API doesn't support accessing files by any method using the inode number. The only access to the inode number is typically through the stat() system call which exposes the inode number, which can be useful for identifying if two filenames are the same file, but is not used for anything else.

Accessing a file by inode would be a security violation, as it would bypass permissions on the directories that contain the file linked to the inode.

The closest you can get to this would be accessing a file by open file handle. But you can't run a program from that either, and this would still require opening the file by a path. (As noted in comments, this functionality was added to linux for security reasons along with the rest of the *at system calls, but is not portable. (yet? standards evolve.))

There's also numerous ways of using the inode number to find the file (basically, crawl the filesystem and use stat) and then run it normally, but this is the opposite of what you want, as it is enormously more expensive than just accessing the file by pathname and doesn't remove that cost either.

Having said that, worrying about this type of optimization is probably moot, as Linux has already optimized the internal inode lookup a great deal. Also, traditionally, shells hash the path location of executables so they don't have to hunt for them from all directories in $PATH every time.

9
  • 1
    Because you are using a file system there need not be any tedious "full string lookups". Nov 20, 2022 at 0:44
  • 15
    Nitpicking: it's possible on Linux to execute a file pointed by a fd through execveat(2) and a mode where dirfd is actually a file fd. For example it's used through the (POSIX) fexecve(3) in this security patch for LXC: github.com/lxc/lxc/commit/…
    – A.B
    Nov 20, 2022 at 11:32
  • 1
    @A.B: ya, I forgot they added that.
    – user10489
    Nov 21, 2022 at 4:49
  • 1
    Some filesystems (including btrfs) have extensions that do allow direct lookup... but that of course requires root for the reasons you mention, and someone would need to write a C module extension to bash for it to be usable from there. Nov 21, 2022 at 15:54
  • 1
    @A.B: It was possible long before then. exec /proc/????/fd/#### worked just fine even if readlink said the path went somewhere that didn't exist because of chrooted into a different area. I used to leave initrd in RAM accessable by /proc/1/fd/10 and had some emergency tools there.
    – Joshua
    Nov 21, 2022 at 17:10
19

Yes, it is possible to execute a file by its inode:

find / -inum 242 -exec {} \; -quit

Performance motivated the question, though, and the above is not performant. Not only is the directory structure walked to find a file having that inode (and there may be multiple), but under the hood, the inode number is resolved to a path, and the path is given to the kernel to execute. But why?

The kernel exposes the exec family of functions (execl, execvp, etc), which all wrap the kernel function execve. That function replaces the current process image with a new process image, one that's been bootstrapped by reading the contents from a given file path. So every way the kernel gives to execute a program requires it be given by path. By using the file path as the entry point, we get all the access control benefits associated with file paths and, for this reason, the "by path" API is the only one in Linux for executing a program.


However, there exists a fiddly and not guaranteed to work in all environments mechanism that allows you to invoke a program from within memory. Since anything in memory is necessarily faster than anything on disk, this drives to the heart of the question: how to run a program as fast as possible.

In early 2002 a (famous) hacker known as grugq introduced the concept of userland exec. This is not a shell's exec function: it's an emulation of every step the kernel's execve function performs, just written in userland. This is ideal for hackers who want to hide their activity because it allows the execution of a program outside the usual access control mechanism of execve.

The implementation for this method requires numerous helpers that can clean the address space, load the dynamic linker if needed, initialize the stack and so on. The mechanism also requires the desired code be loaded in certain kinds of memory.

There are also counter-measures in place to make this kind of thing difficult but, note, not impossible. All that's required is that the target system has page-aligned memory, the ability to mark memory as executable, and the ability to jump to arbitrary points in memory. Those requirements usually translate to: you must write it in C and use it on a system without SELinux or without SELinux being completely enabled. I won't go into the implementation details here, but will provide links that allow you to explore on your own.

So, if your Linux system meets the requirements above, then you can execute code from within memory by:

  1. Loading the code into memory somewhere. Malicious actors will have already side-loaded the desired code into memory as part of the initial drop, but if you wanted to do it along the lines of inode, you could do find / -inum 242 -exec cat {} \;
  2. Invoking the userland exec mechanism, setting its entry point to the address of memory where you stored your program from step 1
  3. Profit

The kernel, filesystem, and shell have all been tuned to make the lookup and execution of programs a negligible fraction of the total overhead necessary to do work. Loading a program in memory and executing it from there is not really in the domain of the average use case, so unless doing this for fun I'd say you'd want to benchmark the performance before investing time in trying.

References:

5
  • 2
    This is a fascinating read, and really illustrates why it's difficult to do. As someone ignorent of these things, I had to look up SELinux. For the record it's Security Enhanced Linux, and it seems to be used by all the big players (e.g. Ubuntu, Fedora, Android).
    – Clumsy cat
    Nov 20, 2022 at 9:00
  • 10
    While user-space execve() is an interesting gimmick, it's really just a red herring in this context. To find the file with the given inode, you now need to do a full search of the directory tree! Granted, you're not doing a name lookup there, but getting all the metadata of all the files in the tree isn't exactly cheap either. Oh and in the end, you need to open the file by name (E.g. that cat {} gives cat the filename, and as another answer says, there's no API to open a file with the inode number only.)
    – ilkkachu
    Nov 20, 2022 at 9:47
  • 5
    In the context of this question, this could be boiled down to find / -inode 242 -exec {} \;. Nov 20, 2022 at 12:58
  • 2
    @StephenKitt The "find by inode" strategy is certainly not what the OP wanted: You end up with a path name as well, which the OP clearly wanted to avoid -- only after a huge delay and circumstance! I suppose the OP has an inode number and wants to basically load the executable from there and execute it, probably close to what bishop described here, but without the find; instead, load the file via your own fs driver, supposedly. Nov 21, 2022 at 16:14
  • 1
    @Peter yes, that was my underlying point — I didn’t mean to imply that this was a valid answer to the question, just that the intersection of the answer and what the question was asking is the find command I gave, not the long explanation of user-space exec. IMO this could be an interesting self-standing Q&A (as long as it included a comparison with what the dynamic loader does), but it’s not an answer to this question. Nov 21, 2022 at 16:31
2

This is not a direct answer to the question about i-nodes, but rather a possible way to avoid looking up the paths of standard utilities in shell scripts.

BusyBox is a program that combines many standard Unix utilities into a single executable that is way smaller than the combined size of all the tools it replaces. It is very popular in the embedded world, where disk size often matters a lot. In a typical BusyBox-based system, sh, ls and grep are all symlinks to busybox. Thus, a shell script that calls ls and grep would only be busybox calling itself twice.

BusyBox has an experimental feature called “standalone shell”. When this is enabled, BusyBox acting as a shell does not perform path lookups for the utilities it implements. Instead, it just executes itself via /proc/self/exe with the correct parameters. For example, if it runs a shell script that calls grep, instead of looking up grep in $PATH, it would execute /proc/self/exe grep <arguments>. There is still a path lookup in the kernel for /proc/self/exe, but it is always the same irrespective of the utility being called, and the executable image is already in memory, so there is no need to load it.

Note, however, that BusyBox was heavily optimized for size rather than for speed, so it may not be your best option if you care about saving a few microseconds. Also, as noted before, the “standalone shell” feature is labeled as experimental.

2
  • 2
    "executable image is already in memory" would be the case regardless. Executables are mmap'd; if the same one is already running, well, it's already mapped (unless the system is under so much memory pressure as to cause parts of it to be swapped out, but that'd be a problem using /proc/self/exe too). Nov 21, 2022 at 14:11
  • This also shows a fundamental problem with the question. If the shell script from the question runs in a BusyBox shell, ls and grep will share the same inode !
    – MSalters
    Nov 22, 2022 at 13:23
-3

is there a way to directly invoke a program in Linux using only its node number

No, it is not, and for a simple reason: you are using the singular form "a program" and "its inode number" in your question, but inodes are not unique: there can be multiple files in the system with the same inode number.

So, by simple common sense, the best you can do is "directly invoke one or more of a set of programs using only their inode numbers", but you cannot invoke a specific program without additional identifying information about said program in order to pick out the one program from the set of programs.

One such identifying information, of course, could be its path … at which point you are back at square one.

6
  • 3
    While multiple filenames can point to the same inode, the inode number of the file itself is still unique, so if you could point to the executable via the inode number, you would get that particular program. Sure, if the program looked at its argv[0] to decide what to do, you'd need to make sure to pass the correct name there. But that's the same as with command line arguments, and on the low level, argv[0] is one of them. It's not tied to the actual path of the file, but passed separately in execve().
    – ilkkachu
    Nov 21, 2022 at 9:06
  • 4
    @ilkkachu: There is no guarantee that inode numbers are unique. They are only guaranteed to be unique per-filesystem. Nov 21, 2022 at 10:18
  • 2
    yes, of course, any reference by inode would necessarily need to include the filesystem.
    – ilkkachu
    Nov 21, 2022 at 11:21
  • @ilkkachu: Indeed, and you identify that filesystem by its mount point, which is a ... path. So instead of a path, you now need a path plus an inode number. That's not making it better, that's making it worse.
    – MSalters
    Nov 22, 2022 at 13:29
  • 1
    @MSalters all mount points also have a device number. If you combine the device number with the inode number, then you do have a unique number kernel-wide that identifies a specific file with a caveat. I believe some of the non-UNIX network file systems can't guarantee a unique inode every time, but I believe all open file handles would operate with a unique one.
    – penguin359
    Nov 22, 2022 at 23:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .