0

I have a small problem related to kernel build time. Sorry for a bit messy writing. I'm very poor at it. :D We (my team) are using qemu for rebuilding Linux kernel for our product. We all share similar hardware - more or less at least (details described later). What's very interesting, the compilation time can differ dramatically between our hosts. We talk about differences between 3-5min till an hour.

Measured after make clean. All of us build it using qemu-arm emulating bash on mounted image of the whole system(system of the device we develop). the system itself doesn't run. Just chroot+emulated arm bash that runs other arm programs and calls kernel of the host. For two of three hosts all of this is additionally encapsulated inside of a full Linux (Ubuntu VM) because they run Windows on hosts. All hosts make use of all available cores.

Timings of full rebuild and description of hardware-software used:

  1. host 1. Win + Hyper-V emulating Ubuntu 20(kernel 5.4.3). I7 8th gen 6 cores. 16GB ram. 8 for VM Time 3m50s.
  2. host 2. Win + VMBox (hardware virtualization) Ubuntu 18(kernel unknown but rather 5+). I7 9th 6 cores. Time around 30 mins. 6GB ram for VM.
  3. host 3. Linux (manjaro). Kernel 5.8. I7 6th 4 cores. 40GB ram. qemu runned on host without Ubuntu VM. Time is about an hour.

Hardware is a bit different but not different enough to generate such differences. Processors are more or less the same. 9th gen goes much worse than 8th. 6th is out of scale. Hardware based(accelerated) virtualization shall not be a problem since host without it makes the worst.(maybe something is different between Linux and Windows? But it doesnt make much sense because at the end of the day it's being build on Linux. Arch/Debian doesn't make difference i guess). Ram shall also not be significant as long as minimum is reached. I think it is in every case.

I write because none of those data makes sense. I was also trying to find what can affect the build time in such configuration but as you see it's quite specific - didn't have much success.

Do you have any idea what specs/settings could also influence the build?

Thanks for your attention,time and bearing my writing style ;-)

4
  • This doesn’t answer the question, but I’m curious as to why you run your kernel build with QEMU instead of cross-compiling (which would be much faster)... – Stephen Kitt Jan 14 at 12:01
  • How do you run make? Do you actually use all the available cores? Specifying the Core naming makes zero sense, there's the Core i7 920 and the i9 10900K where the latter is probably 5-10 times faster. Is your host OS completely idle? What about CPU hardware mitigations? – Artem S. Tashkinov Jan 14 at 13:04
  • @StephenKitt We were actually testing crosscompile with linaro. But this whole build system was meant to build some QT app that was supposed to run atop of that kernel. That was the main goal not the building kernel itself. My coworker was doing it and he found some serious kernel callback problems during builds of QT(I don't have much knowledge about exact cause unfortunately). Second reason was such that with one image we could get build environment and real testing image for arm device. Maybe soon we will switch to buildroot but it's not decided yet. Besides i'm simply purely curious ;-) – user8715154 Jan 14 at 17:40
  • @ArtemS.Tashkinov Yeah... we double checked if all of the cores are being used. Times i gave were on average(except the first one) and we rather build without running anything else except of the minimal of what has to be run (simply OS) At least on those 2 "slower" machines. About core naming, that's why i was writing generations also. And i7 9th gen was doing it half of an hour while 8th 4 mins... older was much faster (Maybe i didn't got you. Correct me if so :-) ) Also the Linux pc is interesting since it emulates qemu only. – user8715154 Jan 14 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.