I am using ARM Foundation Model to emulate a arm-v8 environment(I need both TrustZone and KVM), and I run a kvm-enabled linux on it as the Host OS, the kernel version is 3.14.0(I clone it following the guide on virtualopensystems.
However, when try to run a guest OS, I met with some problems(both kvm-tools and qemu-system-aarch64). Here are what I met:
All the tools and ther kernel source code are from the guide mentioned above, and I use the prebuilt binaries. But I compiled the kernel myself.
KVM Tools:
- I use the prebuilt kvm tools found in the website of virtualopensystems, lkvm-static
- When I boot the guest kernel, it hung at bootconsole [earlycon0] disabled. I did some google work but didn't find not too much. Some say it might because of the wrong devicetree, but I there is no dtb file in the guide.
qemu-system-aarch64
- Also, I use the prebuilt binaries
- When I boot the guest kernel, it seemed 'normal' until it gave messages like the following:
hwclock: can't open '/dev/misc/rtc': No such file or directorymodprobe: can't change directory to '3.14.0': No such file or directoryopenvt: can't open '/dev/tty1': No such file or directory
- The complete log of QEMU would be in the last of this question.
Another thing needs to be mentioned, the .config file of the Host OS is NOT the original one. The original .config file has the KVM feature but lacks the support for DMA-relevant functions, which is needed for my own sake.
So I replaced it with the .config in the kernel of 3.18-rc. Of course, they are not same at all, and when I run make, I have to manually decide some configurations, I decide them all as default. And after compilation, the TrustZone feature is normal, and /dev/kvm is also generated, but when I try to run a guest OS, I met with the problems above.
Could anyone please give me some suggestions?
Thanks a lot.
Tgn Yang
=========================================================
Here are the output when using qemu:
root@FVP:/data/qemu-kvm ./qemu-system-aarch64 --enable-kvm --nographic --kernel
Image --drive if=none,file=disk_oe64.img,id=fs --device virtio-blk-device,drive=fs -m 512 -M virt --cpu host --append "earlyprintk conosole=ttyAMA0 mem=512M rootwait root=/dev/vda rw"
Initializing cgroup subsys cpu
Linux version 3.14.0 (hamayun@hamayun-laptop) (gcc version 4.8.1 (Ubuntu/Linaro4.8.1-10ubuntu7) ) #1 SMP PREEMPT Tue Apr 29 15:37:35 CEST 2014
CPU: AArch64 Processor [410fd000] revision 0
No earlyprintk arguments passed.
Memory limited to 512MB
psci: probing function IDs from device-tree
PERCPU: Embedded 11 pages/cpu @ffffffc01ffe7000 s16128 r8192 d20736 u45056
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129280
Kernel command line: earlyprintk conosole=ttyAMA0 mem=512M rootwait root=/dev/vda rw
PID hash table entries: 2048 (order: 2, 16384 bytes)
Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
software IO TLB [mem 0x5b000000-0x5f000000] (64MB) mapped at [ffffffc01b000000-ffffffc01effffff]
Memory: 436272K/524288K available (4122K kernel code, 357K rwdata, 1748K rodata, 243K init, 284K bss, 88016K reserved)
Virtual kernel memory layout:
vmalloc : 0xffffff8000000000 - 0xffffffbbffff0000 (245759 MB)
vmemmap : 0xffffffbc00e00000 - 0xffffffbc01500000 ( 7 MB)
modules : 0xffffffbffc000000 - 0xffffffc000000000 ( 64 MB)
memory : 0xffffffc000000000 - 0xffffffc020000000 ( 512 MB)
.init : 0xffffffc00063d000 - 0xffffffc000679f00 ( 244 kB)
.text : 0xffffffc000080000 - 0xffffffc00063ca14 ( 5875 kB)
.data : 0xffffffc00067a000 - 0xffffffc0006d37d0 ( 358 kB)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Preemptible hierarchical RCU implementation.
RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=1.
RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS:64 nr_irqs:64 0
Architected cp15 timer(s) running at 100.00MHz (virt).
sched_clock: 56 bits at 100MHz, resolution 10ns, wraps every 2748779069440ns
Calibrating delay loop (skipped), value calculated using timer frequency.. 200.00 BogoMIPS (lpj=1000000)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 1, 8192 bytes)
Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes)
hw perfevents: enabled with arm/armv8-pmuv3 PMU driver, 1 counters available
Brought up 1 CPUs
SMP: Total of 1 processors activated.
devtmpfs: initialized
atomic64 test passed
regulator-dummy: no parameters
NET: Registered protocol family 16
vdso: 2 pages (1 code, 1 data) at base ffffffc000681000
hw-breakpoint: found 16 breakpoint and 16 watchpoint registers.
Serial: AMBA PL011 UART driver
9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 33, base_baud = 0) is a PL011 rev1
console [ttyAMA0] enabled
bio: create slab <bio-0> at 0
SCSI subsystem initialized
Switched to clocksource arch_sys_counter
NET: Registered protocol family 2
TCP established hash table entries: 4096 (order: 3, 32768 bytes)
TCP bind hash table entries: 4096 (order: 4, 65536 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
TCP: reno registered
UDP hash table entries: 256 (order: 1, 8192 bytes)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
kvm [1]: HYP mode not available
futex hash table entries: 256 (order: 2, 16384 bytes)
NFS: Registering the id_resolver key type
Key type id_resolver registered
Key type id_legacy registered
nfs4filelayout_init: NFSv4 File Layout Driver Registering...
fuse init (API version 7.22)
9p: Installing v9fs 9p2000 file system support
msgmni has been set to 852
io scheduler noop registered
io scheduler cfq registered (default)
Serial: AMBA driver
blk-mq: CPU -> queue map
CPU 0 -> Queue 0
vda: unknown partition table
TCP: cubic registered
NET: Registered protocol family 17
9pnet: Installing 9P2000 support
Key type dns_resolver registered
regulator-dummy: disabling
kjournald starting. Commit interval 5 seconds
EXT3-fs (vda): using internal journal
EXT3-fs (vda): mounted filesystem with writeback data mode
VFS: Mounted root (ext3 filesystem) on device 254:0.
devtmpfs: mounted
Freeing unused kernel memory: 240K (ffffffc00063d000 - ffffffc000679000)
INIT: version 2.88 booting
Mounting local filesystems...
Starting udev
udevd[384]: starting version 182
Activating swap
Starting Bootlog daemon: bootlogd.
Initializing random number generator...
random: dd urandom read with 5 bits of entropy available
udev-cache: checking for /dev/shm/udev-regen... found.
Populating dev cache
Populating volatile Filesystems.
Checking for -/run/lock-.
Creating directory -/run/lock-.
Checking for -/var/volatile/log-.
Creating directory -/var/volatile/log-.
Checking for -/var/volatile/tmp-.
Creating directory -/var/volatile/tmp-.
Target already exists. Skipping.
Checking for -/var/lock-.
Creating link -/var/lock- pointing to -/run/lock-.
Checking for -/var/log-.
Creating link -/var/log- pointing to -/var/volatile/log-.
Checking for -/var/run-.
Creating link -/var/run- pointing to -/run-.
Checking for -/var/tmp-.
Creating link -/var/tmp- pointing to -/var/volatile/tmp-.
Checking for -/tmp-.
Creating link -/tmp- pointing to -/var/tmp-.
Checking for -/var/lock/subsys-.
Creating directory -/var/lock/subsys-
Checking for -/var/log/wtmp-.
Creating file -/var/log/wtmp-.
Checking for -/var/run/utmp-.
Creating file -/var/run/utmp-.
Checking for -/etc/resolv.conf-.
Creating link -/etc/resolv.conf- pointing to -/var/run/resolv.conf-.
Checking for -/var/run/resolv.conf-.
Creating file -/var/run/resolv.conf-.
Checking for -/var/log/boot-.
Creating file -/var/log/boot-.
Target already exists. Skipping.
Checking for -/var/run/sepermit-.
Creating directory -/var/run/sepermit-.
Checking for -/var/run/sshd-.
Creating directory -/var/run/sshd-.
Checking for -/var/log/lastlog-.
Creating file -/var/log/lastlog-.
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
System time was Thu Jan 1 00:00:06 UTC 1970.
Setting the System Clock using the Hardware Clock as reference...
hwclock: can't open '/dev/misc/rtc': No such file or directory
System Clock set. System local time is now Thu Jan 1 00:00:06 UTC 1970.
Thu Jun 18 11:03:04 UTC 2015
Saving the System Clock time to the Hardware Clock...
hwclock: can't open '/dev/misc/rtc': No such file or directory
Hardware Clock updated to Thu Jun 18 11:03:04 UTC 2015.
INIT: Entering runlevel: 5
Configuring network interfaces... ifconfig: SIOCGIFFLAGS: No such device
Starting OpenBSD Secure Shell server: sshd
generating ssh RSA key...
generating ssh ECDSA key...
generating ssh DSA key...
generating ssh ED25519 key...
done.
Starting rpcbind daemon...done.
starting statd: done
System time was Thu Jun 18 11:03:36 UTC 2015.
Setting the System Clock using the Hardware Clock as reference...
hwclock: can't open '/dev/misc/rtc': No such file or directory
System Clock set. System local time is now Thu Jun 18 11:03:36 UTC 2015.
modprobe: can't change directory to '3.14.0': No such file or directory
NFS daemon support not enabled in kernel
Starting syslogd/klogd: done
openvt: can't open '/dev/tty1': No such file or directory
Starting auto-serial-console: done
Stopping Bootlog daemon: bootlogd.
INIT: no more processes left in this runlevel
.configfiles are exactly the same, the differences consist of CONFIG_SWAP and other things