15

I love (the way) how Linux & Co. lets users install many packages from different repositories. AFAIK, they come also with source-packages, so you can compile them by yourself.

But why even bother to "keep/offer" pre-compiled packages, when you could just compile them yourself? What are the intentions of keeping/offering them?

Is it possible to configure Linux, to only download source packages and let the OS do the rest? (Just like a pre-compiled package installation?)

Thank you for your answers.

Improve this question
5
  • 19
    I'm not expert in this field, but I would think that it is a service to us, end users. It is much easier and faster to install pre-compiled and tested packages compared to having to compile them.
    – sudodus
    Commented Jul 11, 2022 at 11:55
  • 21
    You might be interested in something like Gentoo. You can compile everything from source and design your own system and compilation options. Just be prepared to spend a lot of time on compilations to keep packages up to date.
    – doneal24
    Commented Jul 11, 2022 at 11:57
  • 26
    actual question on reddit: "Libreoffice has been compiling for 6 hours. Is there any way to speed it up? I need to write a paper in it due tomorrow." - I can see how that's not for everyone :P
    – marcelm
    Commented Jul 12, 2022 at 8:55
  • 17
    Besides the time wasted, think of all the MWh of extra energy that would be needed if everyone was compiling all their software separately (and effect on battery for mobile devices).
    – Stéphane Chazelas
    Commented Jul 12, 2022 at 11:35
  • @StéphaneChazelas: Thanks for your concern about energy consumption. I feel less alone.
    – Eric Duminil
    Commented Jul 14, 2022 at 10:51

5 Answers 5

Reset to default
57

It’s a trade-off: distributions which provide pre-built packages spend the time building them once (in all the configurations they support), and their users can then install them without spending the time to build them. The users accept the distributions’ binaries as-is. If you consider the number of package installations for some of the larger distributions, the time saved by not requiring recompilation everywhere is considerable.

There are some distributions which ship source and the infrastructure required to build it, and rely on users to build everything locally; see for example Gentoo. This allows users to control exactly how their packages are built.

If you go down this path, even with the time savings you can get by simplifying package builds, you should be ready to spend a lot of time building packages. I don’t maintain the most complex packages in Debian, but one of my packages takes over two hours to build on 64-bit x86 builders, and over twelve hours on slower architectures!

Improve this answer
5
  • 3
    Apart from a great answer, thanks for linking that build log. I just scrolled through all of it. Even that took quite a while. It boggles the mind.
    – Oliphaunt
    Commented Jul 11, 2022 at 21:12
  • 8
    @Oliphaunt And GCC is not even one of the particularly bad offenders. LLVM (which has essentially become mandatory for building a GPU accelerated graphical environment on Linux) takes about 10-20% longer in my experience, and most web browsers take more like 50-60% longer at minimum (fun fact, even when throwing all of a Ryzen 9 3950X (32 threads of execution) and 64GB of DDR4-2666 RAM at it, webkit-gtk still takes more than 45 minutes to build).
    – Austin Hemmelgarn
    Commented Jul 11, 2022 at 23:41
  • 14
    The worst offenders are Chromium (over ten hours on x86) and LibreOffice (over six hours on x86), far ahead of any compiler (LLVM 13 takes around three hours, just a few minutes more than GCC 12).
    – Stephen Kitt
    Commented Jul 12, 2022 at 9:35
  • 2
    Apart from the time spent, I don't even want to think about trying to build say Chromium with say only 4GB of memory (and if you have only a small SSD the extra disk space required might be a problem as well).
    – Voo
    Commented Jul 12, 2022 at 9:53
  • 5
    @Voo I still have memories of building X11 (or gcc) from source in the early '90s. If everything went well the process would take 1-2 days. Not a practice that I want to go back to.
    – doneal24
    Commented Jul 12, 2022 at 16:58
26

  1. You imply that all people have enough CPU/RAM/storage/time/knowledge to compile packages. Nope, not the case, the opposite is true, absolute most people don't want to wait e.g. hours to compile something. Firefox compilation on a Raspberry Pi may take several weeks. Is this OK? Nope.

  2. The second reason is that Linux distros build packages in a controlled malware-free properly functioning environment which is not guaranteed for the end user.

  3. All the users of the distro in the end run absolutely the same code which aids in reporting bugs and debugging which might not be true for users deciding to compile software on their own.

  4. Many modern Linux distros support secure boot which requires signing packages with keys which cannot be distributed among end users because it would mean your trust chain is completely broken.

You're welcome to build everything using Gentoo, LFS or the AUR repository in Arch Linux. Actually software packages can be compiled in most distros but the above three were created with compilation in mind.

Improve this answer
19
  • 2
    Arch distributes packages as binaries, unlike Gentoo and LFS.
    – Compizfox
    Commented Jul 12, 2022 at 11:05
  • 1
    @Compizfox I've edited the answer to accommodate your notice.
    – Artem S. Tashkinov
    Commented Jul 12, 2022 at 12:12
  • 2
    Regarding 2, I generally trust the distros I use and their package maintainers, but I would consider user-compiled packages to be strictly safer. If the user is infected, they're already infected. However, leaving the compilation to a distro's maintainers introduces the possibility of someone being a bad actor and including malware in the compiled packages. While I trust both source and pre-built packages to be generally safe, I wouldn't consider pre-built packages to be safer than source packages. The argument for choosing source packages is primarily safety.
    – JoL
    Commented Jul 13, 2022 at 0:08
  • 6
    To give an extreme example of your first point, the oldest computer I've got running Linux is a Pentium MMX. It took four months to compile GCC 6.4; extrapolating from the GCC release timeline and the compilation speed on newer computers, I'd expect GCC 13 (and possibly GCC 14) to be released before it finishes compiling GCC 12.
    – Mark
    Commented Jul 13, 2022 at 0:09
  • 2
    @StephenKitt That doesn't matter from a security perspective. Posting the source together with a binary for security purposes is just a security theater anyway. If you cannot trust the source it doesn't matter if it's just a binary, a binary+sources or just sources. They can all be manipulated. In order to verify such manipulation you need a second trustworthy source in all cases, and at that point just drop the repository and use the source you trust.
    – GACy20
    Commented Jul 13, 2022 at 13:37
20

But why even bother to "keep/offer" pre-compiled packages, when you could just compile them yourself? What are the intentions of keeping/offering them?

Simple economics. Compiling an entire distribution's worth of packages takes weeks, even on a large cluster, uses a lot of energy, and produces a lot of heat.

It simply makes sense to do this only once rather than do it over and over and over and over again for every single user.

It also massively increases the size and complexity (and thus the attack surface!!!) of a basic installation, since you have to include every single compiler for every single programming language that every single package uses as part of the base install.

Many, many years ago, I was really into Linux From Scratch and I wrote a script which automates the entire installation process of a base LFS system. It ran for about two days, and just remember that a base LFS system is a really basic system. It includes the kernel, the libc, the shell, the bootloader, and some basic tools … and that is pretty much it. No graphical environment, no web browser, no email program, no office suite, no multimedia player, no Java environment, no Python / Ruby / PHP / Node.js / whatever your preferred programming language is, no games, no photo editor, no scientific tools, nothing of all of the stuff that actually makes a computer "useful".

And some of these are really big and take a really long time to compile. A single package may easily take weeks to compile, depending on the computer you are running on. (Imagine e.g. your router or your smartwatch.)

Improve this answer
5
  • 1
    On x86, a full archive rebuild of Debian can be done in a couple of days on a single (large enough) system.
    – Stephen Kitt
    Commented Jul 12, 2022 at 9:17
  • 7
    +1 for mentioning the environmental impact.
    – Roger Lipscombe
    Commented Jul 12, 2022 at 12:16
  • 1
    @StephenKitt That's impressively fast. Out of curiosity, what is "large enough" in this context? A normal single socket server with 128GB or the likes?
    – Voo
    Commented Jul 12, 2022 at 15:06
  • 2
    @Voo something like that, yes; I don’t have current figures handy, 128GiB might not be enough, but 256GiB should do it comfortably (with 40-60 cores).
    – Stephen Kitt
    Commented Jul 12, 2022 at 15:31
  • Computers have gotten faster since you did your LFS install. I recently re-compiled my Gentoo system after a CPU upgrade, which took about 14 hours.
    – Mark
    Commented Jul 13, 2022 at 0:12
11

For being a gentoo user since I moved from FreeBSD Pfff! Was it ten years ago, I could not agree more with Stephen's and Artem's answers that I indeed upvote.

I have never been amongst those who believed that setting charming personal options to gcc (thinking of -funroll-loops -fomg-optimize for instance) would lead to significant performances gain despite it has been one of the major reason claimed by a vast majority of compiling addicts.

On my core II duo system… of course… compiling chromium will need more than a full day… : Simply… ridiculous ! Even more just stupid if you want to follow the pace of upstream's updates !

Ridiculous ? Well at the end of the day… not necessarily … depending of what you really care about since there are things you cannot just achieve with pre-built packages !

  • Avoiding pulseaudio with firefox for example.
  • Preferring to rely on your system shared libs rather than on local implementations (harfbuzz, icu, apng, libevent, not to speak of codecs (av1, vpx, webp… (not to speak of proprietary codecs…)))
  • And ultimately… wherever you have enough good reasons to diverge from upstream choices. (your harware becoming obsolete being just an example)

If you don't get any of those wills then… well… simply think that it is obviously not an eco-friendly way to maintain your system… a couple of gentoo lead devs have quit for that reason.

Improve this answer
2
  • 2
    USE flags - the ability to tweak the compile settings of packages (not the compiler) - are indeed a very neat feature in Gentoo.
    – user20574
    Commented Jul 12, 2022 at 9:58
  • 5
    It's been a while but I ran a production HPC cluster based on Gentoo. The major use cases were molecular modeling and sequence analysis. After building everything, including the kernel, with optimizations for the specific chipset in use, I saw about a 4% decrease in runtime. Maybe significant when a job runs for 4 weeks but otherwise not worth the effort.
    – doneal24
    Commented Jul 12, 2022 at 17:04
2

I work in a closed environment. Some of our systems are deemed critical and locked down. Compilers and debuggers are not allowed on the system.
The packages are are installed as-is from the vendor.
Compiling/Installing from source code would:

  1. Allow someone to change the source code, and thereby the functionality of the package in a way that could be unknown to others.
  2. Increase the time required to to build the baseline, promote and install it. In a RTOS environment, outages need to be minimized.
  3. The packages are compiled in the same (standard) way all the time. We don't need to worry about variations in compilers, config files, etc.

    In short, it guarantees an easy, low resource way to insure every computer has exactly the same package installed the way the vendor intends it to be installed.
Improve this answer
2
  • I don't see how this add up beyond what has already been stated. Point 1 doesn't look convincing, you could just don't give permissions (either verbally to subordinates or via the OS) for people to alter a given set of files. Point 2 is covered by Stephen, and 3 by Artem.
    – Quasímodo
    Commented Jul 13, 2022 at 21:49
  • 3
    @Quasímodo You missed the first sentence: "I work in a closed environment". Part of that phrase means: "Things are strictly controlled." Most people are un-aware of these environments. "So, why not compile anyway?" See the 3 points. BTW: Point #1 is MANDATED in the system design documentation. It is technology, policy and procedure that make this happen.
    – Scottie H
    Commented Jul 13, 2022 at 22:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .