Kernel (for 2.6) debugging- kgdb

Kernel (for 2.6) debugging- kgdb &

kgdb is a kernel patch that, once applied, allows for the use of the familiar gdb interface for source-level debugging of a running Linux kernel.

The kgdb patch supplies the kernel with a debugging stub. This stub uses gdb remote serial/Ethernet protocol to communicate with gdb (host machine). This patch is applied to the kernel on the machine that will run the gdb session (host machine).

The gdbmod for module debugging, gdbmod version of gdb is required to debug modules that are loaded dynamically on to the testing machine.

Configuring kgdb

Download kgdb and gdbmod patch for your kernel version from http://kgdb.linsyssoft.com/downloads.htm

Copy the patch to the kernel source directory on the development machine, and apply,

1. Unzip the kgdb patch
   mkdir ${BASE_DIR}/patch-kgdb
   cd ${BASE_DIR}/patch-kgdb
   tar -jxvf linux-2.6.7-kgdb-2.2.tar.bz2
2. patch -p1 < ${BASE_DIR}/patch-kgdb/core-lite.patch
3. patch -p1 < ${BASE_DIR}/patch-kgdb/i386-lite.patch
4. patch -p1 < ${BASE_DIR}/patch-kgdb/8250.patch
5. patch -p1 < ${BASE_DIR}/patch-kgdb/eth.patch
6. patch -p1 < ${BASE_DIR}/patch-kgdb/i386.patch
7. patch -p1 < ${BASE_DIR}/patch-kgdb/core.patch

This kernel will be for target board, the patch applied here are for i386, for ARM patch download latest patch from for kernel 2.6.15.5 http://kgdb.linsyssoft.com/downloads/kgdb-2/linux-2.6.15.5-kgdb-2.4.tar.bz2

Apply the patch for arm board as

1. patch -p1 < ${BASE_DIR}/patch-kgdb/core-lite.patch
2. patch -p1 < ${BASE_DIR}/patch-kgdb/arm-lite.patch
3. patch -p1 < ${BASE_DIR}/patch-kgdb/8250.patch
4. patch -p1 < ${BASE_DIR}/patch-kgdb/eth.patch
5. patch -p1 < ${BASE_DIR}/patch-kgdb/core.patch

The patched kernel must now be recompiled.

        $ make menuconfig

From the main menu navigate to and select kernel hacking. You should now see an option for Remote (serial) debugging with gdb. Make sure this option is selected and then exit.

        $ make clean 

        $ make bzImage

The newly compiled kernel image (e.g., bzImage) is copied to the target machine for booting up.

Compiling the kernel on the development machine

1. In the ${BASE_DIR}/linux-2.6.7/Makefile, set the EXTRAVERSION = -kgdb
2. make mrproper
3. make menuconfig
   Select the options appropriate for the target machine Hardware.
   Select the options pertaining to kgdb under "Kernel hacking" .
   Remote (serial) debugging with gdb
   
4. make clean
5. make bzImage
6. Transfer the built kernel to the Target machine from the Development machine.
   
   Copy the Kernel image from ${BASE_DIR}/linux-2.6.7/arch/i386/boot/bzImage to the target machine as /boot/vmlinuz-2.6.7-kgdb
   Copy the Map file from ${BASE_DIR}/linux-2.6.7/System.map to the target machine as /boot/System.map-2.6.7-kgdb
   Also create links,
   ln -s /boot/vmlinuz-2.6.7-kgdb /boot/vmlinuz
   ln -s /boot/System.map-2.6.7-kgdb /boot/System.map

7. Edit the /boot/grub/grub.conf file in the target machine to have the kgdb enabled kernel entry.
   Sample is shown below :-
   
   # Sample grub.conf which will by default boot the kgdb enabled kernel
   default=1
   timeout=10
   splashimage=(hd0,0)/boot/grub/splash.xpm.gz
   
   title RH
   root (hd0,0)
   kernel /boot/vmlinuz ro root=/dev/sda1
   initrd /boot/initrd
   
   title Linux-2.6.7-kgdb
   root (hd0,0)
   kernel /boot/vmlinuz-2.6.7-kgdb ro root=/dev/sda1 rootfstype=ext3 kgdbwait kgdb8250=0,115200
8. Using kgdb ethernet interface

Add following line in the grub entry :

kgdboe=@192.168.2.150/,@192.168.2.117/ (that's kgdboe=@LOCAL-IP/,@REMOTE-IP/)

# Sample grub.conf which will by default boot the kgdb enabled kernel

title Linux-2.6.15.5-kgdb(eth)

root (hd0,0)

kernel /boot/vmlinuz-2.6.15.5-kgdb ro root=/dev/hda1 kgdboe=@192.168.2.150/,@192.168.2.117/

console=ttyS0,115200

Then for starting the debug session give following command on gdb.

(gdb) ./vmlinux

(gdb) target remote udp:HOSTNAME:6443

Starting the debug session

1. After booting the target machine will wait for the host development machine to connect, by displaying the message:-
   
   Waiting for connection from remote gdb...
2. For version 2.6 this message will not appear, it will just wait after printing message "booting linux, uncompressing linux", can connect to this form host machine,
   
3. cd ${BASE_DIR}/linux-2.6.7
4. For setting a debug session with baud rate of 115200 on /dev/ttyS0 , run as "root" user:-
   
   gdb ./vmlinux
   GNU gdb Red Hat Linux (6.0post-0.20040223.17rh)
   Copyright 2004 Free Software Foundation, Inc.
   GDB is free software, covered by the GNU General Public License, and you are
   welcome to change it and/or distribute copies of it under certain conditions.
   Type "show copying" to see the conditions.
   There is absolutely no warranty for GDB. Type "show warranty" for details.
   This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db library "/lib/tls/libthread_db.so.1".
   
   (gdb) shell echo -e "\003" > /dev/ttyS0
   (gdb) set remotebaud 115200
   (gdb) target remote /dev/ttyS0
   Remote debugging using /dev/ttyS0
   breakpoint () at kernel/kgdb.c:1212
   1212 atomic_set(&kgdb_setting_breakpoint, 0);
   warning: shared library handler failed to enable breakpoint
   (gdb)
5. For further commands refer http://kgdb.linsyssoft.com/tockdebug.htm

Reference

Gdb quick ref guide.

http://users.ece.utexas.edu/~adnan/gdb-refcard.pdf

Simple how to

http://www.shakthimaan.com/downloads/glv/kgdb-howto/kgdb-howto.html

Remote debugging

http://www.linuxjournal.com/article/4525