I want to run two machine:
- debugged machine: compiled kernel with kgdb option.
- debugger machine: for running gdb
How can I debug machine 1 from machine 2?
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It only takes a minute to sign up.Sign up to join this community
Sounds like you're looking for gdbserver.
gdbserver is a control program for Unix-like systems, which allows you to connect your program with a remote GDB via target remote---but without linking in the usual debugging stub.
On the target machine
You need to have a copy of the program you want to debug. gdbserver does not need your program's symbol table, so you can strip the program if necessary to save space. GDB on the host system does all the symbol handling.
target$ gdbserver host:2345 emacs foo.txt
NOTE: You can also attach to running processes like so:
target$ gdbserver comm --attach pid
One the GDB host machine
You need an unstripped copy of your program, since GDB needs symbols and debugging information. Start up GDB as usual, using the name of the local copy of your program as the first argument. (You may also need the
--baud' option if the serial line is running at anything other than 9600bps.) After that, use target remote to establish communications with gdbserver. Its argument is either a device name (usually a serial device, like/dev/ttyb'), or a TCP port descriptor in the form host:PORT. For example:
(gdb) target remote the-target:2345
There is another method discussed in the manuals, called "remote stub". The official manuals are located here, GDB Documentation, on the gnu.org website. Looking through the GDB Users Manual, section 20.5, Implementing a Remote Stub, explains how to use this feature instead of
This method is described as follows in the docs:
The next step is to arrange for your program to use a serial port to communicate with the machine where gdb is running (the host machine). In general terms, the scheme looks like this:
So you might be able to setup a serial port on both the VM host and the guest and debug the guest's kernel using this method.
KGDB + QEMU step-by-step
My QEMU + Buildroot example is a good way to get a taste of it without real hardware: https://github.com/cirosantilli/linux-kernel-module-cheat/tree/1969cd6f8d30dace81d9848c6bacbb8bad9dacd8#kgdb
Pros and cons vs other methods:
The main steps are:
Compile the kernel with:
CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_INFO=y CONFIG_CONSOLE_POLL=y CONFIG_KDB_CONTINUE_CATASTROPHIC=0 CONFIG_KDB_DEFAULT_ENABLE=0x1 CONFIG_KDB_KEYBOARD=y CONFIG_KGDB=y CONFIG_KGDB_KDB=y CONFIG_KGDB_LOW_LEVEL_TRAP=y CONFIG_KGDB_SERIAL_CONSOLE=y CONFIG_KGDB_TESTS=y CONFIG_KGDB_TESTS_ON_BOOT=n CONFIG_MAGIC_SYSRQ=y CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE=0x1 CONFIG_SERIAL_KGDB_NMI=n
Most of those are not mandatory, but this is what I've tested.
Add to your QEMU command:
-append 'kgdbwait kgdboc=ttyS0,115200' \ -serial tcp::1234,server,nowait
Run GDB with from the root of the Linux kernel source tree with:
gdb -ex 'file vmlinux' -ex 'target remote localhost:1234'
and the boot should finish.
echo g > /proc/sysrq-trigger
And GDB should break.
Now we are done, you can use GDB as usual:
b sys_write c
Tested in Ubuntu 14.04.
KGDB + Raspberry Pi
The exact same setup as above almost worked on a Raspberry Pi 2, Raspbian Jessie 2016-05-27.
You just have to learn to do the QEMU steps on the Pi, which are easily Googlable:
add the configuration options and recompile the kernel as explained at https://www.raspberrypi.org/documentation/linux/kernel/building.md There were unfortunately missing options on the default kernel build, notably no debug symbols, so the recompile is needed.
cmdline.txt of the boot partition and add:
gdb to the serial with:
gdb -ex 'file vmlinux' -ex 'target remote /dev/ttyUSB0'
If you are not familiar with the serial, check out this: https://www.youtube.com/watch?v=da5Q7xL_OTo All you need is a cheap adapter like this one. Make sure you can get a shell through the serial to ensure that it is working before trying out KGDB.
echo g | sudo tee /proc/sysrq-trigger
from inside an SSH session, since the serial is already taken by GDB.
With this setup, I was able to put a breakpoint in
sys_write, pause program execution, list source and continue.
However, sometimes when I did
sys_write GDB just hung and printed this error message several times:
Ignoring packet error, continuing...
so I'm not sure if something is wrong with my setup, or if this is expected because of what some background process is doing in the more complex Raspbian image.