I am trying to compile the mainline Linux kernel with a custom config. This one!

Running on a 64 bit system.

At the last step, when linking the Kernel, it fails because it goes OOM (error 137).

  DESCEND objtool
  INSTALL libsubcmd_headers
  CALL    scripts/checksyscalls.sh
  LD      vmlinux.o
make[2]: *** [scripts/Makefile.vmlinux_o:61: vmlinux.o] Error 137
make[2]: *** Deleting file 'vmlinux.o'

ulimit -a says that per process memory is unlimited.

I have tried make, make -j1 make -j4, no difference whatsoever.

Same results with gcc as compiler instead of clang.

Does anyone have a freaking clue on why the compilation eats up so much RAM? It's getting unaffordable to develop Linux :\

  • From the info you give compilation has completed successfully. The killing appears as a matter of fact during the link phase. If you can confirm that this i always the case, please, change "compilation" for "linking" or at least "build" in your message.
    – MC68020
    Commented Sep 1, 2023 at 16:26
  • Also please confirm you are not running a 32bits kernel.
    – MC68020
    Commented Sep 1, 2023 at 16:31
  • Also it seems you are using clang for the build. What happens using good old standard ggc ?
    – MC68020
    Commented Sep 1, 2023 at 16:34
  • 1
    Updated the question as suggested, ty Commented Sep 2, 2023 at 21:34

1 Answer 1


It's getting unaffordable to develop Linux

I am afraid it has always been.
32GB RAM is common on kernel devs desktops. And yet some of them started encountering ooms when building their allyesconfig-ed kernel.

Lucky you… who are apparently not allyesconfig-ing… you should not need more than 32G… ;-)

On a side note, reading CONFIG_HAVE_OBJTOOL=y as part of your .config file, you might take some benefits from the patches submitted as part of the discussion linked hereabove.

Does anyone have a freaking clue on why the compilation eats up so much RAM?

You are probably the only one who could precisely tell. (after considering the size of the miscellaneous *.o files you might be able to find in each top level directory of the kernel source distribution (since compilation was achieved successfuly))

From the information you provide (the kernel.config file) I can only venture a priori :

A/ every component of your kernel will be statically linked : (since I notice that all your selected OPTION_* are marked "=y")
There is nothing wrong with this per se since there can be many good reasons for building everything in-kernel but this will definitely significantly increase the RAM needed when linking all this together.

=> You probably should consider building kernel parts as modules wherever possible.

B/ a good amount of CONFIG_DEBUG appear set. Once again there is nothing wrong with that per se however it is likely to increase significantly the RAM needed to link the different parts, not to say even more since it implies CONFIG_KALLSYMS_*=y

On a side note, considering the debugging feature selected, in addition to CONFIG_HZ_100=y I would assume that you are not searching for best possible latencies / performances. => I would then consider the opportunity to prefer CONFIG_CC_OPTIMIZE_FOR_SIZE

  • Thanks for the info. I managed to build it by switching all loadable to modules instead of built-in with make allmodconfig but now I don't get a console login running it in QEMU (lol) Commented Sep 3, 2023 at 18:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .