How long might it take to build NetBSD userland and kernel?

Question

I am interested to use NetBSD as the operating system on my server. I have not used a system where security updates are performed by source, but have read enough in the guide to feel comfortable trying it. However, I do not know how long this operation is likely to take.

Given a fairly modest server of say 1 processor core and 0.5 to 1.0 GB RAM, how long might it be expected to take to build the userland and kernel of an x86_64 system, following the directions of Chapter 33. Updating an existing system from sources in the guide?

Also, how much local disk space does this operation require? I have not seen mention of that in the guide.

Answer 1

It takes roughly a day for me to build 5 or so different architectures, as complete release builds (aka, tarballs and ISO images.) I'm building on a relatively low memory (512M), i386 virtual machine, using a script that also checks out the sources, and does a bunch of setup around build.sh.

If you have more than one processor/core, you can speed things up by setting the concurrency level to the number of processors/cores, or even as much as twice the number of processors/cores, given the build is predominately I/O bound.

Answer 2

I would suggest that any decent not-so-modern x86_64 true server should be able do a full build in a couple of hours or maybe less, including xsrc.

My NetBSD-current build server is a Xen domU with 8GB RAM and 8 VCPUs running on a Dell PE2950 8-core (Xeon E5440 @2.83GHz) with 32GB RAM and with a decently fast set of SAS disks on the integrated PERC 6/i controller (with the build output going to a RAID-0 partition). That machine only cost me about $650[us], used of course. It can do an NetBSD-5/i386 build of everything to final ISOs, with everything static-linked (i.e. requiring a lot more disk IO and linker memory than a dynamic-linked build), from NFS-mounted sources on another domU on the same server, in less than 2 hours (with -j12). A kernel build (amd64 GENERIC) after a reboot (nothing cached) takes under 5min (with -j12).

At the moment my /build partition has 102GB used and contains objects, binaries, and ISOs for three -current builds (amd64, i386, evbarm) and two 5.x builds (amd64 and i386). Keep in mind that's all separately static-linked binaries -- dynamic-linked builds are much smaller. A static-linked full install (i.e. with xsrc and comp and everything else) takes about 6.6 GB.

Answer 3

Maybe it will take more than 90 min to build the kernel . To speed up the compilation process we need to use -j option for example if you have 2 core you can type:

fakeroot make-kpkg..... -j 2

or

make -j 2

you can speed up the compilation process X2 ( 45 min) , (8 core less than 10min)

Also you can use the CONCURRENCY_LEVEL variable for example if you have 2 core you can type:

export CONCURRENCY_LEVEL=3

generally:

export CONCURRENCY_LEVEL= the number of cores +1

Answer 4

For reference, a ./build.sh -U -j 12 -O /Users/mbelleau/netbsd/obj_amd64 -m amd64 release (no X11) on a MacBook Air i7 1.7GHz 8GB RAM, NetBSD-7.0-release, took roughly 2 hours:

===> Summary of results:
     build.sh command:    ./build.sh -U -j 12 -O /Users/mbelleau/netbsd/obj_amd64 -m amd64 release
     build.sh started:    Thu May 26 23:51:30 EDT 2016
     NetBSD version:      7.0
     MACHINE:             amd64
     MACHINE_ARCH:        x86_64
     Build platform:      Darwin 15.4.0 x86_64
     HOST_SH:             /bin/sh
     No $TOOLDIR/bin/nbmake, needs building.
     Bootstrapping nbmake
     MAKECONF file:       /etc/mk.conf (File not found)
     TOOLDIR path:        /Users/mbelleau/netbsd/obj_amd64/tooldir.Darwin-15.4.0-x86_64
     DESTDIR path:        /Users/mbelleau/netbsd/obj_amd64/destdir.amd64
     RELEASEDIR path:     /Users/mbelleau/netbsd/obj_amd64/releasedir
     Created /Users/mbelleau/netbsd/obj_amd64/tooldir.Darwin-15.4.0-x86_64/bin/nbmake
     Updated makewrapper: /Users/mbelleau/netbsd/obj_amd64/tooldir.Darwin-15.4.0-x86_64/bin/nbmake-amd64
     Successful make release
     build.sh ended:      Fri May 27 01:43:58 EDT 2016
===> .

And for space usage:

$ du -sh /Users/mbelleau/netbsd/*
129M    /Users/mbelleau/netbsd/gnusrc.tgz
3.7G    /Users/mbelleau/netbsd/obj_amd64
7.1M    /Users/mbelleau/netbsd/sharesrc.tgz
137M    /Users/mbelleau/netbsd/src.tgz
 48M    /Users/mbelleau/netbsd/syssrc.tgz
2.9G    /Users/mbelleau/netbsd/usr
158M    /Users/mbelleau/netbsd/xsrc.tgz

The subsequent (updates) release builds should be quicker; I updated using CVS to the 7.0-stable tag (cvs update -r netbsd-7-0 -Pd) and rebuilt, took roughly an hour:

===> Summary of results: 
     build.sh command:    ./build.sh -U -u -j 12 -O /Users/mbelleau/netbsd/obj_amd64 -m amd64 release
     build.sh started:    Fri May 27 09:08:10 EDT 2016
     NetBSD version:      7.0.1_PATCH
     MACHINE:             amd64
     MACHINE_ARCH:        x86_64
     Build platform:      Darwin 15.4.0 x86_64
     HOST_SH:             /bin/sh
     No $TOOLDIR/bin/nbmake, needs building.
     Bootstrapping nbmake
     MAKECONF file:       /etc/mk.conf (File not found)
     TOOLDIR path:        /Users/mbelleau/netbsd/obj_amd64/tooldir.Darwin-15.4.0-x86_64
     DESTDIR path:        /Users/mbelleau/netbsd/obj_amd64/destdir.amd64
     RELEASEDIR path:     /Users/mbelleau/netbsd/obj_amd64/releasedir
     Created /Users/mbelleau/netbsd/obj_amd64/tooldir.Darwin-15.4.0-x86_64/bin/nbmake
     Updated makewrapper: /Users/mbelleau/netbsd/obj_amd64/tooldir.Darwin-15.4.0-x86_64/bin/nbmake-amd64
     Successful make release
     build.sh ended:      Fri May 27 10:11:14 EDT 2016
===> .