2

I’m trying to make a big endian build of a linux distribution for ARM. Since I’m on Gentoo, cross-compiling couldn’t be easier. I’ve already built it all, but then got stuck with getting it/the kernel to boot.

I’m targetting Cubieboard with AllWinner A10 CPU. As a bootloader I use u-boot. Since u-boot doesn’t support big endian ARM, I patched it exactly before passing control to the kernel:

diff -Naur u-boot-2016.01-1/arch/arm/lib/bootm.c u-boot-2016.01-2/arch/arm/lib/bootm.c
--- u-boot-2016.01-1/arch/arm/lib/bootm.c       2016-01-12 15:06:54.000000000 +0100
+++ u-boot-2016.01-2/arch/arm/lib/bootm.c       2017-07-09 14:13:29.675865446 +0200
@@ -315,7 +315,16 @@
                                                          0, machid, r2);
                } else
 #endif
+               {
+                       {
+                               unsigned long v;
+                               __asm volatile ("mrc p15, 0, %0, c1, c0, 0\n\t"
+                                               "orr %0, %0, #(1 << 7)\n\t" /* Switch to bigendian */
+                                               "mcr p15, 0, %0, c1, c0, 0" : "=&r" (v));
+                       }
+
                        kernel_entry(0, machid, r2);
+               }
        }
 #endif
 }

I originally used a little different syntax, but in the end took a known-to-work code from APEX boot loader (its arm-kernel-shim). (Nonetheless, the register is the same as what I had and what I read in ARM documentation.)

Also, since u-boot needs to be little endian, I prepared another toolchain for it—with target arm-linux-gnueabihf. As far as I can tell, u-boot itself boots fine.

The (mainline) kernel is compiled with a toolchain for target armeb-linux-gnueabihf. From the compiled image (arch/arm/boot/Image within kernel source/build tree) I built a bootable image (using mkimage from my u-boot build):

mkimage -C none -A arm -T kernel -n Linux-4.9.9-gentoo -d /usr/armeb-linux-gnueabihf/usr/src/linux/arch/arm/boot/Image -ep 0x48000000 -a 0x48000000 /usr/armeb-linux-gnueabihf/boot/uimage

I took the DTB file also from kernel:

cp /usr/armeb-linux-gnueabihf/usr/src/linux/arch/arm/boot/dts/sun4i-a10-cubieboard.dtb /usr/armeb-linux-gnueabihf/boot/

When I load it all up to µSD card and try to boot, I get the following output on serial console:

U-Boot SPL 2016.01 (Jul 16 2017 - 13:52:00)
DRAM: 1024 MiB
CPU: 1008000000Hz, AXI/AHB/APB: 3/2/2
Trying to boot from MMC


U-Boot 2016.01 (Jul 16 2017 - 13:52:00 +0200) Allwinner Technology

CPU:   Allwinner A10 (SUN4I)
I2C:   ready
DRAM:  1 GiB
MMC:   SUNXI SD/MMC: 0
In:    serial
Out:   serial
Err:   serial
SCSI:  SUNXI SCSI INIT
SATA link 0 timeout.
AHCI 0001.0100 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
flags: ncq stag pm led clo only pmp pio slum part ccc apst
Net:   eth0: ethernet@01c0b000
starting USB...
USB0:   USB EHCI 1.00
USB1:   USB OHCI 1.0
USB2:   USB EHCI 1.00
USB3:   USB OHCI 1.0
scanning bus 0 for devices... 1 USB Device(s) found
scanning bus 2 for devices... 1 USB Device(s) found
Hit any key to stop autoboot:  0
=> setenv bootargs console=tty0 console=ttyS0,115200 earlyprintk hdmi.audio=EDID:0 disp.screen0_output_mode=EDID:1280x800p60 root=PARTUUID=AC9D6C6F-01 rootwait panic=10
=> ext2load mmc 0 0x48000000 boot/uimage
5025856 bytes read in 592 ms (8.1 MiB/s)
=> ext2load mmc 0 0x51000000 boot/sun4i-a10-cubieboard.dtb
28542 bytes read in 237 ms (117.2 KiB/s)
=> bootm 0x48000000 - 0x51000000
## Booting kernel from Legacy Image at 48000000 ...
   Image Name:   Linux-4.9.9-gentoo
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    5025792 Bytes = 4.8 MiB
   Load Address: 48000000
   Entry Point:  48000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 51000000
   Booting using the fdt blob at 0x51000000
   Loading Kernel Image ... OK
   Loading Device Tree to 49ff6000, end 49ffff7d ... OK

Starting kernel ...

And it just hangs there like this – no progress, no output, nothing. My question is, how to proceed from here/how to find out what is actually happening? Am I missing something, did I do anything wrong (or didn’t do something)?

Some more things I tried, but without success:

  • word-swapping the kernel image (as APEX does) (resulted in undefined instruction on boot),

  • using compressed kernel image,

  • using legacy FEX file instead of FDT.


Update 2017/07/21: I’ve been partly successful in solving my issue. I took the hint from Tom Rini’s comment and tried packing zImage into the uImage which made the kernel boot. With a custom init program (just a simple Hello World compiled either BE or LE) I confirmed the other Tom Rini’s suspicion: my original kernel was not big endian, but little. To fix this, I added the following two lines to the top of kernel’s .config:

CONFIG_ARCH_SUPPORTS_BIG_ENDIAN=y
CONFIG_CPU_BIG_ENDIAN=y

I sourced my inspiration from a how-to for Nvidia Jetson TK1. I also added the following line to arch/arm/mach-sunxi/Kconfig at the end of A10 section:

select ARCH_SUPPORTS_BIG_ENDIAN

It turned out that this is enough to build and boot a big endian kernel as the kernel itself switches the CPU to big endian mode. However, it does so with setend instruction---its scope is only for the kernel itself, not for userspace (as link in Murray Jensen’s answer explains).

I’m going to try:

  • jumping to kernel’s decompressor entry point via exception, or (if I fail)

  • patching the kernel so that it spawns processes into big endian settings (although I sort of smell a minefield there...).

  • Did you confirm that the resulting zImage is big endian? And please note that uImages are made from the zImage, not Image, see arch/arm/boot/Makefile in the kernel for more details. – Tom Rini Jul 18 '17 at 12:15
  • Would the FDT blob need to be big endian also? or is the endianness of this fixed by a standard somewhere? (where?) – Murray Jensen Jul 20 '17 at 3:35
  • @TomRini Thank you, this has been very helpful! I thought the kernel is BE since I compiled it with BE compiler, but it apparently wasn’t enough. I found a page that describes compiling BE kernel for a different platform and used hints from there. Also, I tried zImage previously, but failed, still I gave it a try again now. And now I boot BE. :-) What I can’t yet, is to run the userspace (Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b), but it’s already a great progress! – xHire Jul 21 '17 at 19:49
  • @MurrayJensen It seems like it’s either fixed or the kernel can handle both. What I can say is that the FDT blob didn’t recompile when I forced the kernel into BE mode. And both LE and BE kernels boot with the same version of the file. – xHire Jul 21 '17 at 19:51
0

Don't know a lot about ARM (yet), but according to this, "The EE bit in the CP15 System Control Register (SCR) determines the endianness set on exception (i.e. the endianness of the OS itself)." ... so it seems to me that you need to enter the loaded kernel image via an exception somehow i.e. you can't just jump to the address (as U-Boot appears to do - see boot_jump_linux() in this).

  • Note that the zImage/uImage doesn't start the kernel itself, rather it starts the decompression wrapper that in turn starts the kernel. That, however never starts in this case. – Tom Rini Jul 18 '17 at 12:13
  • Yes, so the decompression wrapper would need to be big endian also ... and you would have to jump to it via an exception after setting the EE bit. – Murray Jensen Jul 20 '17 at 3:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.