Different methods to run a non-nixos executable on Nixos

Question

What are the different methods to run a non-nixos executable on NixOs? (For instance proprietary binaries.) I'd like to see also the manual methods.

Answer 1

Short version

Quick and dirty: make sure steam-run is installed (strange name, it has nothing to do with steam), e.g. nix-shell -p steam-run, then:

$ steam-run ./your-binary

Here is a longer and more detailed explanation, together with various methods, often less dirty. Since the creation of this answer, other alternatives to steam-run have been developed, see e.g. nix-ld (that basically recreates the missing loaders in /lib) and nix-alien / nix-autobahn that also automatically try to add the missing libraries. Finally you can use distrobox that provides you any distribution in a docker/podman container tightly integrated with the host.

Long version

Here are several methods (the manual ones are mostly for educational purpose as most of the time writing a proper derivation is better). I'm not an expert at all, and I did this list also to learn nix, so if you have better methods, let me know!

So the main issue is that the executable call first a loader, and then needs some libraries to work, and nixos put both the loader and the libraries in /nix/store/.

This list gives all the methods I found so far. There are basically three "groups":

• the full manual: interesting for educational purpose, and to understand what's going on, but that's all (don't use them in practice because nothing will prevent the derivations used to be garbage collected later)
• the patched versions: these methods try to modify the executable (automatically when using the recommended method 4 with autoPatchelfHook) to make the point to the good library directly
• the methods based on FHS, that basically fake a "normal linux" (more heavy to run than the patched version, so this should be avoided if possible).

I would recommend method 4 with autoPatchelfHook for a real, proper setup, and if you don't have time and just want to run a binary in one-line, you may be interested by the quick-and-dirty solution based on steam-run (method 7).

Method 1) Dirty manual method, no patch

You need to first find the loader with for example file:

$ file wolframscript
wolframscript: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, BuildID[sha1]=079684175aa38e3633b60544681b338c0e8831e0, stripped

Here the loader is /lib64/ld-linux-x86-64.so.2. To find the loader of nixos, you can do:

$ ls /nix/store/*glibc*/lib/ld-linux-x86-64.so.2
/nix/store/681354n3k44r8z90m35hm8945vsp95h1-glibc-2.27/lib/ld-linux-x86-64.so.2

You also need to find to find the libraries that your program require, for example with ldd or LD_DEBUG=libs:

$ ldd wolframscript
        linux-vdso.so.1 (0x00007ffe8fff9000)
        libpthread.so.0 => /nix/store/sw54ph775lw7b9g4hlfvpx6fmlvdy8qi-glibc-2.27/lib/libpthread.so.0 (0x00007f86aa321000)
        librt.so.1 => /nix/store/sw54ph775lw7b9g4hlfvpx6fmlvdy8qi-glibc-2.27/lib/librt.so.1 (0x00007f86aa317000)
        libdl.so.2 => /nix/store/sw54ph775lw7b9g4hlfvpx6fmlvdy8qi-glibc-2.27/lib/libdl.so.2 (0x00007f86aa312000)
        libstdc++.so.6 => not found
        libm.so.6 => /nix/store/sw54ph775lw7b9g4hlfvpx6fmlvdy8qi-glibc-2.27/lib/libm.so.6 (0x00007f86aa17c000)
        libgcc_s.so.1 => /nix/store/sw54ph775lw7b9g4hlfvpx6fmlvdy8qi-glibc-2.27/lib/libgcc_s.so.1 (0x00007f86a9f66000)
        libc.so.6 => /nix/store/sw54ph775lw7b9g4hlfvpx6fmlvdy8qi-glibc-2.27/lib/libc.so.6 (0x00007f86a9dae000)
        /lib64/ld-linux-x86-64.so.2 => /nix/store/sw54ph775lw7b9g4hlfvpx6fmlvdy8qi-glibc-2.27/lib64/ld-linux-x86-64.so.2 (0x00007f86aa344000)

Here, you see that most libraries are found except libstdc++.so.6. So let's find them! A first quick and dirty way to find them is to check if they are already present in your system:

$ find /nix/store -name libstdc++.so.6
/nix/store/12zhmzzhrwszdc8q3fwgifpwjkwi3mzc-gcc-7.3.0-lib/lib/libstdc++.so.6

In case the library is not already installed, you will certainly prefer to use the more involved nix-index program to search for these files in a much larger database (thanks hydra). For that, first install nix-index and generate the database (this is only needed the first time, but it can take a few minutes to run):

$ nix-index

(you can also use nix-index-update from nix-alien to download the cache automatically from nix-index-database) then, to search for a library you can do something like (note that --top-level removes some entries):

$ nix-locate lib/libstdc++.so.6 --top-level
…
gcc-unwrapped.lib                                     0 s /nix/store/7fv9v6mnlkb4ddf9kz1snknbvbfbcbx0-gcc-10.3.0-lib/lib/libstdc++.so.6
…

Then you can install these libraries for this quick and dirty example (later we will see better solutions).

Good. Now, we just need to run the program with the LD_LIBRARY_PATH configured to point to this file (see also makeLibraryPath to generate this string in a derivation), and call the loader we determined at the first step on this file:

LD_LIBRARY_PATH=/nix/store/12zhmzzhrwszdc8q3fwgifpwjkwi3mzc-gcc-7.3.0-lib/lib/:$LD_LIBRARY_PATH /nix/store/681354n3k44r8z90m35hm8945vsp95h1-glibc-2.27/lib/ld-linux-x86-64.so.2 ./wolframscript

(make sure to use ./ before the script name, and to keep only the directory of the libraries. If you have several libraries, just use concat the path with colons)

Method 2) Dirty manual method, with patch

After installing (with nixenv -i or in your configuration.nix) patchelf, you can also directly modify the executable to pack the good loader and libraries. To change the loader just run:

patchelf --set-interpreter /nix/store/681354n3k44r8z90m35hm8945vsp95h1-glibc-2.27/lib/ld-linux-x86-64.so.2 wolframscript

and to check:

$ patchelf --print-interpreter wolframscript
/nix/store/681354n3k44r8z90m35hm8945vsp95h1-glibc-2.27/lib/ld-linux-x86-64.so.

and to change the path to the libraries hardcoded in the executable, first check what is the current rpath (empty for me):

$ patchelf --print-rpath wolframscript

and append them to the library path you determined before, eventually separated with colons:

$ patchelf --set-rpath /nix/store/12zhmzzhrwszdc8q3fwgifpwjkwi3mzc-gcc-7.3.0-lib/lib/ wolframscript
$ ./wolframscript

Method 3) Patch in a nix derivation

We can reproduce more or less the same thing in a nix derivation inspired by skypeforlinux

This example presents also an alternative, either you can use:

patchelf --set-interpreter ${glibc}/lib/ld-linux-x86-64.so.2 "$out/bin/wolframscript" || true

(which should be pretty clear once you understand the "manual" method), or

patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" "$out/bin/wolframscript" || true

This second method is a bit more subtle, but if you run:

$ nix-shell '<nixpkgs>' -A hello --run 'echo $NIX_CC/nix-support/dynamic-linker "->" $(cat $NIX_CC/nix-support/dynamic-linker)'
/nix/store/8zfm4i1aw4c3l5n6ay311ds6l8vd9983-gcc-wrapper-7.4.0/nix-support/dynamic-linker -> /nix/store/sw54ph775lw7b9g4hlfvpx6fmlvdy8qi-glibc-2.27/lib/ld-linux-x86-64.so.2

you will see that the file $NIX_CC/nix-support/dynamic-linker contains a path to the loader ld-linux-x86-64.so.2.

Put in derivation.nix, this is

{ stdenv, dpkg,glibc, gcc-unwrapped }:
let

  # Please keep the version x.y.0.z and do not update to x.y.76.z because the
  # source of the latter disappears much faster.
  version = "12.0.0";

  rpath = stdenv.lib.makeLibraryPath [
    gcc-unwrapped
    glibc
  ];
  # What is it for?
  # + ":${stdenv.cc.cc.lib}/lib64";

  src = ./WolframScript_12.0.0_LINUX64_amd64.deb;

in stdenv.mkDerivation {
  name = "wolframscript-${version}";

  system = "x86_64-linux";

  inherit src;

  nativeBuildInputs = [
  ];

  buildInputs = [ dpkg ];

  unpackPhase = "true";

  # Extract and copy executable in $out/bin
  installPhase = ''
    mkdir -p $out
    dpkg -x $src $out
    cp -av $out/opt/Wolfram/WolframScript/* $out
    rm -rf $out/opt
  '';

  postFixup = ''
    # Why does the following works?
    patchelf --set-interpreter "$(cat $NIX_CC/nix-support/dynamic-linker)" "$out/bin/wolframscript" || true
    # or
    # patchelf --set-interpreter ${glibc}/lib/ld-linux-x86-64.so.2 "$out/bin/wolframscript" || true
    patchelf --set-rpath ${rpath} "$out/bin/wolframscript" || true
  '';

  meta = with stdenv.lib; {
    description = "Wolframscript";
    homepage = https://www.wolfram.com/wolframscript/;
    license = licenses.unfree;
    maintainers = with stdenv.lib.maintainers; [ ];
    platforms = [ "x86_64-linux" ];
  };
}

and in default.nix put:

{ pkgs ? import <nixpkgs> {} }:

pkgs.callPackage ./derivation.nix {}

Compile and run with

nix-build
result/bin/wolframscript

Method 4) Use autoPatchElf: simpler

All the previous methods need a bit of work (you need to find the executables, patch them...). NixOs did for us a special "hook" autoPatchelfHook that automatically patches everything for you! You just need to specify it in (native)BuildInputs, and nix does the magic.

{ stdenv, dpkg, glibc, gcc-unwrapped, autoPatchelfHook }:
let

  # Please keep the version x.y.0.z and do not update to x.y.76.z because the
  # source of the latter disappears much faster.
  version = "12.0.0";

  src = ./WolframScript_12.0.0_LINUX64_amd64.deb;

in stdenv.mkDerivation {
  name = "wolframscript-${version}";

  system = "x86_64-linux";

  inherit src;

  # Required for compilation
  nativeBuildInputs = [
    autoPatchelfHook # Automatically setup the loader, and do the magic
    dpkg
  ];

  # Required at running time
  buildInputs = [
    glibc
    gcc-unwrapped
  ];

  unpackPhase = "true";

  # Extract and copy executable in $out/bin
  installPhase = ''
    mkdir -p $out
    dpkg -x $src $out
    cp -av $out/opt/Wolfram/WolframScript/* $out
    rm -rf $out/opt
  '';

  meta = with stdenv.lib; {
    description = "Wolframscript";
    homepage = https://www.wolfram.com/wolframscript/;
    license = licenses.mit;
    maintainers = with stdenv.lib.maintainers; [ ];
    platforms = [ "x86_64-linux" ];
  };
}

Method 5) Use FHS to simulate a classic linux shell, and manually execute the files

Some sofware may be hard to package that way because they may heavily rely on the FHS file tree structure, or may check that the binary are unchanged. You can then also use buildFHSUserEnv to provide an FHS file structure (lightweight, using namespaces) for your application. Note that this method is heavier that the patch-based methods, and add significant startup time, so avoid it when possible

You can either just spawn a shell and then manually extract the archive and execute the file, or directly package your program for the FHS. Let's first see how to get a shell. Put in a file (say fhs-env.nix) the following:

let nixpkgs = import <nixpkgs> {};
in nixpkgs.buildFHSUserEnv {
   name = "fhs";
   targetPkgs = pkgs: [];
   multiPkgs = pkgs: [ pkgs.dpkg ];
   runScript = "bash";
}

and run:

nix-build fhs-env.nix
result/bin/fhs

You will then get a bash in a more standard-looking linux, and you can run commands to run your executable, like:

mkdir wolf_fhs/
dpkg -x WolframScript_12.0.0_LINUX64_amd64.deb wolf_fhs/
cd wolf_fhs/opt/Wolfram/WolframScript/bin/
./wolfram

If you need more libraries/programs as dependencies, just add them to multiPkgs (for all supported archs) or targetPkgs (for current arch only).

Bonus: you can also launch a fhs shell with a one line command, without creating a specifc file:

nix-build -E '(import <nixpkgs> {}).buildFHSUserEnv {name = "fhs";}' && ./result/bin/fhs

Method 6) Use FHS to simulate a classic linux shell, and pack the files inside

source: https://reflexivereflection.com/posts/2015-02-28-deb-installation-nixos.html

Method 7) steam-run

With buildFHSUserEnv you can run lot's of softwares, but you will need to specify manually all the required libraries. If you want a quick solution and you don't have time to check precisely what are the required libraries, you may want to try steam-run (despite the name, it is not linked directly with steam, and just packs lots of libraries), which is like buildFHSUserEnv with lot's of common libraries preinstalled (some of them may be non-free like steamrt that packs some nvidia code, thanks simpson!). To use it, just install steam-run, and then:

steam-run ./wolframscript

or if you want a full shell:

steam-run bash

Note that you may need to add nixpkgs.config.allowUnfree = true; (or whitelist this specific package) if you want to install it with nixos-rebuild, and if you want to run/install it with nix-shell/nix-env you need to put { allowUnfree = true; } in ~/.config/nixpkgs/config.nix.

It is not easy to "overwrite" packages or libraries to nix-shell, but if you want to make a wrapper around your script, you can either manually create a wrapper script:

#!/usr/bin/env nix-shell
#!nix-shell -i bash -p steam-run
exec steam-run ./wolframscript "$@"

or directly write it in a nixos derivation :

{ stdenv, steam-run, writeScriptBin }:
let
  src = ./opt/Wolfram/WolframScript/bin/wolframscript;
in writeScriptBin "wolf_wrapped_steam" ''
    exec ${steam-run}/bin/steam-run ${src} "$@"
  ''

or if you start from the .deb (here I used makeWrapper instead):

{ stdenv, steam-run, dpkg, writeScriptBin, makeWrapper }:
stdenv.mkDerivation {
  name = "wolframscript";
  src = ./WolframScript_12.0.0_LINUX64_amd64.deb;

  nativeBuildInputs = [
    dpkg makeWrapper
  ];
  unpackPhase = "true";
  installPhase = ''
    mkdir -p $out/bin
    dpkg -x $src $out
    cp -av $out/opt/Wolfram/WolframScript/bin/wolframscript $out/bin/.wolframscript-unwrapped
    makeWrapper ${steam-run}/bin/steam-run $out/bin/wolframscript --add-flags $out/bin/.wolframscript-unwrapped
    rm -rf $out/opt
  '';
}

(if you are too tired to write the usual default.nix, you can run directly nix-build -E "with import <nixpkgs> {}; callPackage ./derivation.nix {}")

Method 8) Using nix-ld

If you do not want to spawn a sandbox as we did for steam-run (in sandboxes it's impossible to run setuid apps, sandboxes can't be nested, poor integration with the system packages included direnv), you can recreate the missing loader system-wide by putting in your configuration.nix:

  programs.nix-ld.enable = true;

You can see that the file is now present:

$ ls /lib64/
ld-linux-x86-64.so.2

However it is still impossible to run binaries as the new ld-linux-x86-64.so.2 file only redirects to the loader in NIX_LD (this way multiple programs can use different loaders while being on the same system):

$ ./blender 
cannot execute ./blender: NIX_LD or NIX_LD_x86_64-linux is not set

To locally create this environment variable, you can do something like:

$ cat shell.nix 
with import <nixpkgs> {};
mkShell {
  NIX_LD_LIBRARY_PATH = lib.makeLibraryPath [
    stdenv.cc.cc
    openssl
    # ...
  ];
  NIX_LD = lib.fileContents "${stdenv.cc}/nix-support/dynamic-linker";
}

$ nix-shell 

[nix-shell:/tmp/blender-3.2.2-linux-x64]$ ./blender

or system-wide using:

  environment.variables = {
      NIX_LD_LIBRARY_PATH = lib.makeLibraryPath [
        pkgs.stdenv.cc.cc
        pkgs.openssl
        # ...
      ];
      NIX_LD = lib.fileContents "${pkgs.stdenv.cc}/nix-support/dynamic-linker";
  };

Note that you need to restart your X11 session everytime you change this file or do:

$ cat /etc/profile | grep set-environment
. /nix/store/clwf7wsykkjdhbd0v8vb94pvg81lnsba-set-environment
$ . /nix/store/clwf7wsykkjdhbd0v8vb94pvg81lnsba-set-environment

Note that (contrary to steam-run) nix-ld does not come with any library by default but you can add your own or use tools to do that automatically, see below. You can also get inspired by the list of libraries that steam-run packs here: https://github.com/NixOS/nixpkgs/blob/master/pkgs/games/steam/fhsenv.nix Here is for example the file I'm using for now, it is enough to run blender:

  # Automatically creates a loader in /lib/* to avoid patching stuff
  programs.nix-ld.enable = true;
  environment.variables = {
      NIX_LD_LIBRARY_PATH = with pkgs; lib.makeLibraryPath [
        stdenv.cc.cc
        openssl
        xorg.libXcomposite
        xorg.libXtst
        xorg.libXrandr
        xorg.libXext
        xorg.libX11
        xorg.libXfixes
        libGL
        libva
        pipewire.lib
        xorg.libxcb
        xorg.libXdamage
        xorg.libxshmfence
        xorg.libXxf86vm
        libelf
        
        # Required
        glib
        gtk2
        bzip2
        
        # Without these it silently fails
        xorg.libXinerama
        xorg.libXcursor
        xorg.libXrender
        xorg.libXScrnSaver
        xorg.libXi
        xorg.libSM
        xorg.libICE
        gnome2.GConf
        nspr
        nss
        cups
        libcap
        SDL2
        libusb1
        dbus-glib
        ffmpeg
        # Only libraries are needed from those two
        libudev0-shim
        
        # Verified games requirements
        xorg.libXt
        xorg.libXmu
        libogg
        libvorbis
        SDL
        SDL2_image
        glew110
        libidn
        tbb
        
        # Other things from runtime
        flac
        freeglut
        libjpeg
        libpng
        libpng12
        libsamplerate
        libmikmod
        libtheora
        libtiff
        pixman
        speex
        SDL_image
        SDL_ttf
        SDL_mixer
        SDL2_ttf
        SDL2_mixer
        libappindicator-gtk2
        libdbusmenu-gtk2
        libindicator-gtk2
        libcaca
        libcanberra
        libgcrypt
        libvpx
        librsvg
        xorg.libXft
        libvdpau
        gnome2.pango
        cairo
        atk
        gdk-pixbuf
        fontconfig
        freetype
        dbus
        alsaLib
        expat
      ];
      NIX_LD = lib.fileContents "${pkgs.stdenv.cc}/nix-support/dynamic-linker";
  };  

You can also find the name of the libraries, see above nix-index. You can also use nix-alien-ld or nix-autobahn to automatically find and load the libraries for you. Note that if you don't have the right libraries you will get an error like

$ ./blender
./blender: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory

You can see at once all the libraries that are not yet available using:

$ LD_LIBRARY_PATH=$NIX_LD_LIBRARY_PATH ldd turtl
        libpangocairo-1.0.so.0 => /nix/store/n9h110ffps25rdkkim5k802p3p5w476m-pango-1.50.6/lib/libpangocairo-1.0.so.0 (0x00007f02feb83000)
        libatk-1.0.so.0 => not found
…

Method 9) nix-alien

nix-alien automatically builds a FHS with the appropriates libraries. If you have flake enabled (otherwise just add replace nix run with nix --extra-experimental-features "nix-command flakes" run) you can simply run it this way (nix-alien is not yet packaged in 2022)

nix run github:thiagokokada/nix-alien -- yourprogram

It will then automatically find the library using nix-index, asking you some questions when it is not sure (this is cached).

Note that programs that rely on openGl need to use nixGl to run (this certainly apply to other methods here):

nix run --impure github:guibou/nixGL --override-input nixpkgs nixpkgs/nixos-21.11 -- nix run github:thiagokokada/nix-alien -- blender

Note that you may need to change the version of nixos-21.11 to ensure that the version of openGl matches your program.

Note that you can also see the automatically generated file (the path is given the first time the program is run):

$ cat /home/leo/.cache/nix-alien/87a5d119-f810-5222-9b47-4809257c60ec/fhs-env/default.nix
{ pkgs ? import <nixpkgs> { } }:

let
  inherit (pkgs) buildFHSUserEnv;
in
buildFHSUserEnv {
  name = "blender-fhs";
  targetPkgs = p: with p; [
    xorg.libX11.out
    xorg.libXfixes.out
    xorg.libXi.out
    xorg.libXrender.out
    xorg.libXxf86vm.out
    xorg_sys_opengl.out
  ];
  runScript = "/tmp/blender-3.2.2-linux-x64/blender";
}

See also the other version working with nix-ld and nix-autobahn.

Method 9) Using containers/Docker (heavier)

TODO

Note that the project distrobox allows you to simply create new containers tightly integrated with the host installing any distribution you want.

Method 10) Rely on flatpack/appimage

https://nixos.org/nixos/manual/index.html#module-services-flatpak

appimage-run : To test with, ex, musescore

Sources or examples

• https://github.com/NixOS/nixpkgs/blob/5a9eaf02ae3c6403ce6f23d33ae569be3f9ce644/pkgs/applications/video/lightworks/default.nix
• https://sandervanderburg.blogspot.com/2015/10/deploying-prebuilt-binary-software-with.html
• https://github.com/NixOS/nixpkgs/blob/35c3396f41ec73c5e968a11f46d79a94db4042d7/pkgs/applications/networking/dropbox/default.nix

Also, for people wanting to get started in packaging, I wrote recently a similar tutorial here.