31

I'm a long time Linux user for over 15 years but one thing I hate with a passion is the mandated directory structure. I don't like that /usr/bin is the dumping ground for binaries or libs in /usr/lib, /usr/lib32, /usr/libx32, /lib, /lib32 etc... Random stuff in /usr/share etc. It's dumb and confusing. But some like it and tastes differ.

I want a directory structure where each package is isolated. Imagine instead if the media player dragon had it's own structure:

/software/dragon
/software/dragon/bin/x86/dragon
/software/dragon/doc/README
/software/dragon/doc/copyright
/software/dragon/lib/x86/libdragon.so

Or:

/software/zlib/include/zlib.h
/software/zlib/lib/1.2.8/x86/libz.so
/software/zlib/lib/1.2.8/x64/libz.so
/software/zlib/doc/examples/...
/software/zlib/man/...

You get the point. What are my options? Is there any Linux distro that uses something like my scheme? Can some distro be modified to work like I want it (Gentoo??) or do I need LFS? Is there any prior art in this area? Like publications on if the scheme is feasible or unfeasible?

Not looking for OS X. :) But OS X-inspired is totally ok.

Edit: I have no idea how PATH, LD_LIBRARY_PATH and other environment variables that depend on a small set of paths should work out. I'm thinking that if I have the KDE editor Kate installed in /software/kate/bin/x86/bin/kate then I'm ok with having to type the full path to the binary to start it. How it should work for dynamic libraries and dlopen calls, I don't know but it can't be an unsolvable engineering problem.

  • 3
    What would your search path look like if all your binaries are hidden all over the place? How do you update the path in already running processes/sessions when you install software after their start? What would your security measures be to prevent an average user from somehow managing to modify a binary in some obscure bin-branch? You certainly loose the comfort of possibly making /usr a read-only parition, possibly a common net-mount for a lot of machines ... – Hagen von Eitzen Sep 5 '15 at 11:37
  • 1
    @HagenvonEitzen But (using the OP's naming scheme and examples) you could make /software read-only instead, for the same benefits and drawbacks as making /usr read-only in FHS. – a CVn Sep 5 '15 at 13:42
  • Dynamic libraries can use install names or RPATHs. – asmeurer Sep 5 '15 at 17:58
  • 1
    Your question reminded me of cr.yp.to/slashpackage.html. Not sure this is relevent, though. – bli Sep 9 '15 at 7:59
50

First, an up-front conflict-of-interest disclaimer: I am a long-time GoboLinux developer.

Second, an up-front claim of domain expertise: I am a long-time GoboLinux developer.

There are a few different structures in current use. GoboLinux has one, and tools like GNU Stow, Homebrew, etc, use something quite similar (primarily for user programs). NixOS also uses a non-standard hierarchy for programs, and philosophy of life. It's also a reasonably common LFS experiment.

I'm going to describe all of those, and then comment from experience on how that works out in practice ("feasibility"). The short answer is that yes, it's feasible, but you have to really want it.


GoboLinux

GoboLinux has a structure very similar to what you describe. Software is installed under /Programs: /Programs/ZSH/5.0.8 contains all the files belonging to ZSH 5.0.8, in the usual bin/lib/... directories. The system tools create symlinks to those files under a /System/Links hierarchy, which maps onto /usr¹. The PATH variable contains only the single unified executable directory, and LD_LIBRARY_PATH is unused. Multiple versions of software can coexist at once, but only one file by a given name (bin/zsh) will be linked actively at once. You can access the others by their full paths.

A set of compatibility symlinks also exists, so /bin and /usr/bin map to the unified executables directory, and so on. This makes life easier for software at run time. A kernel patch, GoboHide, allows those compatibility symlinks to be hidden from file listings (but still traversable).

Contra another answer, you do not need to modify kernel code: GoboHide is purely cosmetic, and the kernel does not depend on user-space paths in general². GoboLinux does have a bespoke init system, but that is also not required to do this.

The tagline has always been "the filesystem is the package manager", but there are reasonably ordinary package-management tools in the system. You can do everything using cp, rm, and ln, though.

If you want to use GoboLinux, you are very welcome. I will note, though, that it's a small development team, and you're likely to find that some software you want isn't packaged up if nobody has wanted to use it before. The good news is that it's generally fairly easy to build a program for the system (a standard "recipe" is about three lines long); the bad news is that sometimes it's unpleasantly complicated, which I'll cover more below.

Publications

There are a few "publications". I gave a presentation at linux.conf.au 2010 on the system as a whole that covers everything generally, which is available in video: ogv mp4 (also on your local Linux Australia mirror); I also wrote up my notes into prose. There are also a few older documents, including the famous "I am not clueless", on the GoboLinux website, which addresses some objections and issues. I think that we're all a bit less gung-ho these days, and I suspect that a future release will adopt /usr as the base location for the symlinks.


NixOS

NixOS puts each installed program into its own directory under /nix/store. Those directories are named something like /nix/store/5rnfzla9kcx4mj5zdc7nlnv8na1najvg-firefox-3.5.4/ — there is a cryptographic hash representing the whole set of dependencies and configuration leading to that program. Inside that directory are all the associated files, with more-or-less normal locations locally.

It also allows you to have multiple versions around at once, and to use any of them. NixOS has a whole philosophy associated with it of reproducible configuration: it's essentially got a configuration management system baked into it from the start. It relies on some environmental manipulation to present the right world of installed programs to the user.


LFS

It's fairly straightforward to go through Linux From Scratch and set up exactly the hierarchy you want: just make the directories and configure everything to install in the right place. I've done it a few times in building GoboLinux experiments, and it's not substantially harder than plain LFS. You do need to make the compatibility symlinks in that case; otherwise it is substantially harder, but careful use of union mounts could probably avoid that if you really wanted.

I feel like there was an LFS Hint about exactly that at one point, but I can't seem to find it now.


On Feasibility

The thing about the FHS is that it's a standard, it's very common, and it broadly reflects the existing usage at the time it was written. Most users will never be on a system that doesn't follow that layout in essence. The result of that is that lots of software has latent dependencies on it that nobody realises, often completely unintentionally.

All those scripts with #!/bin/bash? No good if you don't have Bash there. That is why GoboLinux has all those compatibility symlinks; it's just practical. A lot of software fails to function either at build time or at run time under a non-standard layout, and then it requires patching to correct, often quite intrusively.

Your basic Autoconf program will usually happily install itself wherever you tell it, and it's fairly easy to automate the process of passing in the correct --prefix. Other build systems aren't always so nice, either by intentionally baking in the hierarchy, or by leading authors to write non-portable configuration. CMake is a major offender in the latter category. That means that if you want to live in this world you have to be prepared to do a lot of fiddly work up front in other people's build systems. It is a real hassle to have to dynamically patch generated files during compilation.

Runtime is another matter again. Many programs have assumptions about where their own files, or someone else's files, are found either relative to them or absolutely. When you start using symlinks to present a consistent view, lots of programs have bugs handling them (or sometimes, arguably correct behaviour that is unhelpful to you). For example, a tool foobar may expect to find the baz executable next to it, or in ../sbin. Depending on whether it reads its symlink or not, those can be two different places, and neither of them may be correct anyway.

A combined problem is the /usr/share directory. It's for shared files, of course, but when you put every program in its own prefix they're no longer actually shared. That leads to programs unable to find standard icons and the like. GoboLinux dealt with this in a pretty ugly way: at build time, $prefix/share was a symlink to $prefix/Shared, and after building the link was pointed to the global share directory instead. It now uses compile-time sandboxing and file movement to deal with share (and the other directories), but runtime errors from reading links can still be an issue.

Suites of multiple programs are another problem. GoboLinux has never gotten GNOME working fully, and I don't believe NixOS has either, because the layout interdependencies are so baked in that it's just intractable to cure them all.

So, yes, it's feasible, but:

  • There is quite a lot of work involved in just making things function.
  • Some software may just never work.
  • People will look at you funny.

All of those may or may not be a problem for you.


¹ Version 14.01 uses /System/Index, which maps directly onto /usr. I suspect a future version may drop the Links/Index hierarchy and use /usr across the board.

² It does require /bin/sh to exist by default.

  • 1
    Great answer! Are software authors mostly receptive to patches to make them work in gobolinux or hostile to it? – Björn Lindqvist Sep 5 '15 at 16:24
  • Most of the time upstreamed patches go in fine, but sometimes the maintainers can be a bit belligerent about relying on the FHS. I also remember the KDE maintainers insisting that copying a symlink to an unmodifiable template file instead of dereferencing it to copy the file was by design. A lot of patches are from one hard-coded path to another, rather than a proper fix, so they aren't worth sending upstream anyway. – Michael Homer Sep 5 '15 at 21:06
  • So if you found and fund(in time or money) the creation/forking/patching of all the software you want/need, (eh hem, like Apple/Microsoft does/appears to do), then your golden? That's what it sounds like to me. I'm too interested in norm-breaking innovations that aren't hostile-to-desktop-radical like this. – ThorSummoner Sep 9 '15 at 16:23
  • 2
    @ThorSummoner: Most distributions patch all their software heavily; that "funding" is already reality. Those patches are a mix of functional and, yes, path changes to match the distribution's particularities. Of course, as an end user, you don't really notice it, but it's there - there's a lot of labour behind a distribution that it's easy to take for granted. By and large you don't need to patch things - only 13% of GoboLinux recipes involve any sort of patch at all, for example, which is actually lower than, say, Debian - though it's an annoying sort of patching to do when it is required. – Michael Homer Sep 9 '15 at 20:54
6

Both GoboLinux (which F.sb mentioned) and GNU guix are distributions that use a per-package directory structure along with symlinks to point to the "current" version of a binary.

GoboLinux seems to be the better bet if you want a stable system. GNU guix explicitly says it's not yet production-ready. GoboLinux has been around for years. I've never tried either myself.

5

check the GoboLinux.

if you want the directory structure to be changed you should change some kernel code , boot process, directory runlevels based on rc files , and package manager, then the directory structure.

5

Linux FHS is based on what Sun and other UNIX companies decided in the late 1980s.

An important change at that time was to abandon /usr/local/ and to introduce /opt//{bin!lib!man!...}

If you are looking for the reason why /usr/bin today is used as a dumping ground, I believe that GNOME is one of the most responsible projects.

What happened with the 32 bit vs. 64 bit libraries seems to be caused by FHS.

Solaris introduced platform specific subdirectories in /lib /usr/bin and /usr/lib. Do your wish is how Sun enhanced the base concept from 1988.

  • I don't see what you mean by saying "abandon /usr/local". FHS specifically mentions /usr/local as well as /opt. – a CVn Sep 5 '15 at 13:44
  • 2
    The original FHS developed by Sun, HP, IBM, AT&T, SGI of course removed the non-systematic and problematic /usr/local. I have no idea why the Linux people reintroduced that mistake. – schily Sep 5 '15 at 16:15
  • 2
    "Do your wish is how Sun enhanced the base concept from 1988." I don't understand this sentence. – Faheem Mitha Oct 30 '15 at 11:19
4

If every package had its own part of the file system, you'd need extremely large and unwieldy environment variables PATH, LD_LIBRARY_PATH and similar.

You can of course install packages yourself this way, and then use something like GNU Modules to manage whether they're in your environment or not, and this is what we do for scientific software where I work, but we don't do it for system software.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.