1

I have been having problems compiling a router's kernel for QEMU. I have the router working in QEMU using an OpenWRT kernel, but networking does not work. This is why I want to compile the original kernel.

The below command is the problematic command that the (main) Makefile indirectly executes. I say indirectly because it doesn't even explicitly choose to execute the configure script, it just chooses to do so because it is in the directory of downloaded packages that are needed to compile the kernel.

PATH=/home/debian/build-new/host/bin:/home/debian/build-new/host/usr/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games AR="/usr/bin/ar" AS="/usr/bin/as" LD="/usr/bin/ld" NM="/usr/bin/nm" CC="/home/debian/build-new/host/usr/bin/ccache /usr/bin/gcc" GCC="/home/debian/build-new/host/usr/bin/ccache /usr/bin/gcc" CXX="/home/debian/build-new/host/usr/bin/ccache /usr/bin/g++" CPP="/usr/bin/cpp" CPPFLAGS="-I/home/debian/build-new/host/usr/include" CFLAGS="-O2 -I/home/debian/build-new/host/usr/include" CXXFLAGS="-O2 -I/home/debian/build-new/host/usr/include" LDFLAGS="-L/home/debian/build-new/host/lib -L/home/debian/build-new/host/usr/lib -Wl,-rpath,/home/debian/build-new/host/usr/lib" PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG="/home/debian/build-new/host/usr/bin/pkg-config" PKG_CONFIG_SYSROOT_DIR="/" PKG_CONFIG_LIBDIR="/home/debian/build-new/host/usr/lib/pkgconfig:/home/debian/build-new/host/usr/share/pkgconfig" PERLLIB="/home/debian/build-new/host/usr/lib/perl" LD_LIBRARY_PATH="/home/debian/build-new/host/usr/lib:" CFLAGS="-O2 -I/home/debian/build-new/host/usr/include" LDFLAGS="-L/home/debian/build-new/host/lib -L/home/debian/build-new/host/usr/lib -Wl,-rpath,/home/debian/build-new/host/usr/lib" CC="/usr/bin/gcc" ./configure --prefix="/home/debian/build-new/host/usr" --sysconfdir="/home/debian/build-new/host/etc" --enable-shared --disable-static --disable-gtk-doc --disable-doc --disable-docs --disable-documentation --with-xmlto=no --with-fop=no ccache_cv_zlib_1_2_3=no

The flag that breaks the command is LDFLAGS.

LD="/usr/bin/ld" LDFLAGS="-L/home/debian/build-new/host/lib -L/home/debian/build-new/host/usr/lib -Wl,-rpath,/home/debian/build-new/host/usr/lib" LDFLAGS="-L/home/debian/build-new/host/lib -L/home/debian/build-new/host/usr/lib -Wl,-rpath,/home/debian/build-new/host/usr/lib"

The output of running the command is:

debian@debian-i686:~/build-new/build/host-ccache-3.1.8$ PATH=/home/debian/build-new/host/bin:/home/debian/build-new/host/usr/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games AR="/usr/bin/ar" AS="/usr/bin/as" LD="/usr/bin/ld" NM="/usr/bin/nm" CC="/home/debian/build-new/host/usr/bin/ccache /usr/bin/gcc" GCC="/home/debian/build-new/host/usr/bin/ccache /usr/bin/gcc" CXX="/home/debian/build-new/host/usr/bin/ccache /usr/bin/g++" CPP="/usr/bin/cpp" CPPFLAGS="-I/home/debian/build-new/host/usr/include" CFLAGS="-O2 -I/home/debian/build-new/host/usr/include" CXXFLAGS="-O2 -I/home/debian/build-new/host/usr/include" LDFLAGS="-L/home/debian/build-new/host/lib -L/home/debian/build-new/host/usr/lib -Wl,-rpath,/home/debian/build-new/host/usr/lib" PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG="/home/debian/build-new/host/usr/bin/pkg-config" PKG_CONFIG_SYSROOT_DIR="/" PKG_CONFIG_LIBDIR="/home/debian/build-new/host/usr/lib/pkgconfig:/home/debian/build-new/host/usr/share/pkgconfig" PERLLIB="/home/debian/build-new/host/usr/lib/perl" LD_LIBRARY_PATH="/home/debian/build-new/host/usr/lib:" CFLAGS="-O2 -I/home/debian/build-new/host/usr/include" LDFLAGS="-L/home/debian/build-new/host/lib -L/home/debian/build-new/host/usr/lib -Wl,-rpath,/home/debian/build-new/host/usr/lib" CC="/usr/bin/gcc" ./configure --prefix="/home/debian/build-new/host/usr" --sysconfdir="/home/debian/build-new/host/etc" --enable-shared --disable-static --disable-gtk-doc --disable-doc --disable-docs --disable-documentation --with-xmlto=no --with-fop=no ccache_cv_zlib_1_2_3=no
configure: WARNING: unrecognized options: --enable-shared, --disable-static, --disable-gtk-doc, --disable-doc, --disable-docs, --disable-documentation, --with-xmlto, --with-fop
configure: Configuring ccache
checking for gcc... /usr/bin/gcc
checking whether the C compiler works... no
configure: error: in `/home/debian/build-new/build/host-ccache-3.1.8':
configure: error: C compiler cannot create executables
See `config.log' for more details

Removing only the LDFLAGS fixes the particular error, but then I have another error later.

debian@debian-i686:~/build-new/build/host-ccache-3.1.8$ PATH=/home/debian/build-new/host/bin:/home/debian/build-new/host/usr/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games AR="/usr/bin/ar" AS="/usr/bin/as" LD="/usr/bin/ld" NM="/usr/bin/nm" CC="/home/debian/build-new/host/usr/bin/ccache /usr/bin/gcc" GCC="/home/debian/build-new/host/usr/bin/ccache /usr/bin/gcc" CXX="/home/debian/build-new/host/usr/bin/ccache /usr/bin/g++" CPP="/usr/bin/cpp" CPPFLAGS="-I/home/debian/build-new/host/usr/include" CFLAGS="-O2 -I/home/debian/build-new/host/usr/include" CXXFLAGS="-O2 -I/home/debian/build-new/host/usr/include" PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG="/home/debian/build-new/host/usr/bin/pkg-config" PKG_CONFIG_SYSROOT_DIR="/" PKG_CONFIG_LIBDIR="/home/debian/build-new/host/usr/lib/pkgconfig:/home/debian/build-new/host/usr/share/pkgconfig" PERLLIB="/home/debian/build-new/host/usr/lib/perl" LD_LIBRARY_PATH="/home/debian/build-new/host/usr/lib:" CFLAGS="-O2 -I/home/debian/build-new/host/usr/include" CC="/usr/bin/gcc" ./configure --prefix="/home/debian/build-new/host/usr" --sysconfdir="/home/debian/build-new/host/etc" --enable-shared --disable-static --disable-gtk-doc --disable-doc --disable-docs --disable-documentation --with-xmlto=no --with-fop=no ccache_cv_zlib_1_2_3=no
configure: WARNING: unrecognized options: --enable-shared, --disable-static, --disable-gtk-doc, --disable-doc, --disable-docs, --disable-documentation, --with-xmlto, --with-fop
configure: Configuring ccache
checking for gcc... /usr/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... configure: error: in `/home/debian/build-new/build/host-ccache-3.1.8':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

Removing all flags (to get the command below) allows the configure script to function perfectly.

./configure --prefix="/home/debian/build-new/host/usr" --sysconfdir="/home/debian/build-new/host/etc" --enable-shared --disable-static --disable-gtk-doc --disable-doc --disable-docs --disable-documentation --with-xmlto=no --with-fop=no ccache_cv_zlib_1_2_3=no

What the configure script is crashing on is it is trying to find the files path/to/lib/libc.so.0 and path/to/usr/lib/uclibc_nonshared.a. The problem is, the script is trying to find these libraries in /lib/ and /usr/lib/ even though the Makefile explicitly sets where it is supposed to get the libraries from. Manually symlinking the libraries from where LDFLAGS points to have it link to /lib/ and /usr/lib/ only results in the message:

/usr/bin/ld: skipping incompatible /lib/libc.so.0 when searching for /lib/libc.so.0
/usr/bin/ld: cannot find /lib/libc.so.0
/usr/bin/ld: skipping incompatible /usr/lib/uclibc_nonshared.a when searching for /usr/lib/uclibc_nonshared.a
/usr/bin/ld: cannot find /usr/lib/uclibc_nonshared.a

Also, setting the LD flag to LD="/home/debian/build-new/host/usr/bin/mips-linux-ld" does not fix the problem.

How do I get the Makefile to compile properly? I left some logs and configs on Github's gist service.

Edit:

Using @filbranden's tip I have now reached the point of getting results such as the below output:

/home/debian/build-new/toolchain/gcc-4.7.3-intermediate/./gcc/xgcc -B/home/debian/build-new/toolchain/gcc-4.7.3-intermediate/./gcc/ -B/home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/bin/ -B/home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/lib/ -isystem /home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/include -isystem /home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/sys-include -g -Os -O2 -g -Os -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -isystem ./include -fPIC -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector -fPIC -I. -I. -I../.././gcc -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc/. -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc/../gcc -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc/../include -DHAVE_CC_TLS -o _fractHADI_s.o -MT _fractHADI_s.o -MD -MP -MF _fractHADI_s.dep -DSHARED -DL_fract -DFROM_HA -DTO_DI -c /home/debian/build-new/toolchain/gcc-4.7.3/libgcc/fixed-bit.c

/home/debian/build-new/toolchain/gcc-4.7.3-intermediate/./gcc/xgcc -B/home/debian/build-new/toolchain/gcc-4.7.3-intermediate/./gcc/ -B/home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/bin/ -B/home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/lib/ -isystem /home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/include -isystem /home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/sys-include -g -Os -O2 -g -Os -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -isystem ./include -fPIC -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector -fPIC -I. -I. -I../.././gcc -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc/. -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc/../gcc -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc/../include -DHAVE_CC_TLS -o _fractHATI_s.o -MT _fractHATI_s.o -MD -MP -MF _fractHATI_s.dep -DSHARED -DL_fract -DFROM_HA -DTO_TI -c /home/debian/build-new/toolchain/gcc-4.7.3/libgcc/fixed-bit.c

/home/debian/build-new/toolchain/gcc-4.7.3-intermediate/./gcc/xgcc -B/home/debian/build-new/toolchain/gcc-4.7.3-intermediate/./gcc/ -B/home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/bin/ -B/home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/lib/ -isystem /home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/include -isystem /home/debian/build-new/host/usr/mips-buildroot-linux-uclibc/sys-include -g -Os -O2 -g -Os -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition -isystem ./include -fPIC -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector -fPIC -I. -I. -I../.././gcc -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc/. -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc/../gcc -I/home/debian/build-new/toolchain/gcc-4.7.3/libgcc/../include -DHAVE_CC_TLS -o _fractHASF_s.o -MT _fractHASF_s.o -MD -MP -MF _fractHASF_s.dep -DSHARED -DL_fract -DFROM_HA -DTO_SF -c /home/debian/build-new/toolchain/gcc-4.7.3/libgcc/fixed-bit.c

This compilation has been running for the past 17-18 hours now (and has not crashed or done anything else to indicate that an error may have occurred). It does seem a bit weird that it is still working on fixed-bit.c, but maybe that's normal?

  • I believe -Wl,-rpath,/home/debian/build-new/host/usr/lib, should be -Wl,-rpath=/home/debian/build-new/host/usr/lib, but I'm not positive that it is what is breaking your build. – Lucas Ramage Feb 12 at 3:05
  • I just tried that change. It did nothing to help with fixing it. I am trying out a x86-i686 build of uclibc from toolchains.bootlin.com/releases_x86-i686.html. Trying the two files allowed me to get new errors (of missing files) instead of the ones in the question. I am currently working on a debian package to install to the system. I will create an answer if it works. – SenorContento Feb 12 at 3:15
  • Nope, I now have undefined references to /usr/lib/gcc/i686-linux-gnu/6/../../../i386-linux-gnu/Scrt1.o: In function '_start':. – SenorContento Feb 12 at 3:21
  • 1
    Looks like you're using the wrong compiler... Do you have another gcc somewhere under build-new that you think might be part of the right toolchain that can build binaries for your target? – filbranden Feb 12 at 10:48
  • 1
    @SenorContento Yeah, sounds like you should be using that one instead of /usr/bin/gcc, maybe try setting the $CC and similar variables to point to that one instead. – filbranden Feb 12 at 20:50
0

Using @filbranden's comment, I was able to compile the kernel for my router (there are more error's that need to be solved, but that isn't the scope of this question).

I left a log and my config of what I was doing to compile the kernel on Github gist (new logs and config). The config is broken and won't be apparent until the stage of actually compiling linux, but those solutions are simple.

I ran the below commands to compile the kernel:

make menuconfig O=~/build-new/

RANLIB="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-ranlib" READELF="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-readelf" OBJDUMP="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-objdump" AR="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-ar" AS="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-as" LD="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-gcc" NM="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-nm" CC="/home/debian/bin-new/ccache-3.1.8/ccache /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-gcc" GCC="/home/debian/bin-new/ccache-3.1.8/ccache /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/gcc" CXX="/home/debian/bin-new/ccache-3.1.8/ccache /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/g++" CPP="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-cpp" CPPFLAGS="-I/home/debian/build-new/host/usr/include" CFLAGS="-O2 -I/home/debian/build-new/host/usr/include" CXXFLAGS="-O2 -I/home/debian/build-new/host/usr/include" LDFLAGS="-L/home/debian/build-new/host/lib -L/home/debian/build-new/host/usr/lib" make autoconf O=~/build-new/

RANLIB="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-ranlib" READELF="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-readelf" OBJDUMP="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-objdump" AR="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-ar" AS="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-as" LD="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-gcc" NM="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-nm" CC="/home/debian/bin-new/ccache-3.1.8/ccache /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-gcc" GCC="/home/debian/bin-new/ccache-3.1.8/ccache /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/gcc" CXX="/home/debian/bin-new/ccache-3.1.8/ccache /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/g++" CPP="/home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-cpp" CPPFLAGS="-I/home/debian/build-new/host/usr/include" CFLAGS="-O2 -I/home/debian/build-new/host/usr/include" CXXFLAGS="-O2 -I/home/debian/build-new/host/usr/include" LDFLAGS="-L/home/debian/build-new/host/lib -L/home/debian/build-new/host/usr/lib -Wl,-rpath,/home/debian/build-new/host/usr/lib" make O=~/build-new/

Given the below error, I had to still symlink some binaries before running the make command:

Checking for C compiler ... none found
ERROR: no C compiler found
Checking for linker ... '/home/debian/build-new/host/usr/bin/mips-buildroot-linux-uclibc-gcc' not found (user)
ERROR: no linker found
Checking for ar ... /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-ar ()
Checking for ranlib ... /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-ranlib ()
Checking for readelf ... none found
ERROR: no readelf found
Checking for objdump ... /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-objdump ()

The binaries I symlinked are below. I put a pound sign in front of the ones I don't think matter if symlinked (because the path was never able to be set from prefixing to the make command).

mkdir -p /home/debian/build-new/host/usr/bin/
cp -r /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/* /home/debian/build-new/host/usr/bin/
cd /home/debian/build-new/host/usr/bin/

ln -s mips-linux-uclibc-gcc mips-buildroot-linux-uclibc-gcc
#ln -s /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-ranlib mips-buildroot-linux-uclibc-ranlib
#ln -s /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-readelf mips-buildroot-linux-uclibc-readelf
#ln -s /home/debian/new-kernel/sagem/uclibc-crosstools-gcc-4.2.3-3/usr/bin/mips-linux-uclibc-objdump mips-buildroot-linux-uclibc-objdump

The config and commands here are still broken when it comes to building the kernel itself, but they function well enough to get past the stage of this error. I managed to successfully compile the kernel this morning (took more than 24 hours of pure compile time), but after booting it in QEMU and mounting my filesystem I copied from my router, I came to realize I chose the wrong byte order (I chose LSB instead of MSB by choosing big endian instead of little endian).

Otherwise, I have successfully compiled the kernel using @filbranden's help.

  • I actually looked through my menuconfig to fix the byte order. Funny thing, I chose the right byte order. Choosing little endian has the makefile wanting to use mipsel-...-gcc instead of mips-...-gcc. The compiler generated little endian when I chose big endian. I am now on a quest to figure out why. Other than being compiled for the wrong byte order, the kernel compiled perfectly. I just cannot use the router's programs (that I got from piping the rootfs over netcat on ethernet) because the programs are big endian and I have a little endian kernel. – SenorContento Feb 14 at 22:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.