BDM is way cool, especially if you are bringing up a custom board and don't want to invest in an In Circuit Emulator. You should definitely include a BDM connector on any custom hardware, and all off-the-shelf boards have it. For a broad introduction, see: http://www.macraigor.com/zenofbdm.pdf
Be careful to ensure that the CPU watchdog is disabled in your BDM probes' configuration file, otherwise it will continually reset the CPU and nothing will work. In particulary, the watchdog is enabled at reset and must be disabled if you wish to single step from reset. Also check that DER is zero'd when running with the debugger. Leaving the BDM probe connected can interfere with the target if it is not configured absolutely correctly, which usually involves the BDM probe being completely passive once the kernel is running. Symptoms are that the kernel crashes with the BDM probe connected, but runs fine without it. If you are using BDM and experience unexpected kernel crashes, try disconnecting the BDM probe.
If you're using an off-the-shelf board, you generally won't need to use BDM at all, as you can get by fine with just the serial console. Some developers find BDM invaluable; to others it's more trouble than it's worth. One advantage of a BDM interface over a kernel debugger is that you can really "freeze" the CPU including all timers, interrupts etc.
Some BDM debuggers are capable of performing MMU table walks, which is essential for debugging in virtual memory environments such as Linux because otherwise the BDM port merely deals with raw physical addresses. However, many BDM systems can't do this, so check with the vendor that they support MMU table walks on the particular CPU you're interested in before committing to one; otherwise it will be almost impossible to use once the kernel has turned the MMU on. See: http://lists.linuxppc.org/listarcs/linuxppc-embedded/199909/msg00021.html.
Once the kernel is running you can use gdb remotely or even run it natively on the target hardware if you have enough RAM. It may be worth providing extra RAM on some of your development boards to allow for this. See below.
See "Using GDB for Remote Debugging: BDM Support" in the CrossGCC FAQ at: http://www.objsw.com/CrossGCC/FAQ-7.html
BDM debugging under GDB is not supported by all BDM hardware vendors. See the thread: http://www.oarcorp.com/rtems/freesupport/rtemslistArchive/1999/july/msg00057.html
To support BDM debugging under gdb requires the appropriate remote-* device in GDB, and possibly a kernel driver. If you are cross-developing, you must configure gdb as follows to include the appropriate devices:
% configure --target=powerpc-linux
The following systems currently support BDM debugging on gdb:
http://www.vas-gmbh.de/software/mpcbdm/
Build your own. Supports extensible Flash programming and implements software-tablewalk so you have MMU support; you can use GDB to debug kernel code (single stepping, breakpoints, etc.) and/or inspect kernel data.
http://www.abatron.ch/BDI/bdimain.html
Has an ideal BDM/JTAG emulation solution for Linux hosting, with Linux MMU support specifically in mind. It is an Ethernet-based unit which has a telnet interface with which one can program various popular flash parts. This is ideal for Linux since one can build a kernel and expect script the programming process automagically.
Their external box implements the standard GDB remote protocol allowing you to host debugging on any platform that can host GDB.
http://cyclone.parad.ru/ppcdbm/
Essentially an earlier version of MPCBDM, only of historical interest now.
Patches to add support to gdb are available in: ftp://ftp.hmi.com/pub/gdb/
Currently only support Windoze platform, but say Linux support for the Kestrel device (but not the Wiggler, Raven, etc) should be available sometime in September 2000.
OCD Commander is a free assembly level debugger for Windows, which may work under VMWare. I couldn't get it to work with the Raven/Blackbird interface on the Embedded Planet CLLF under Windows; it wouldn't single step correctly.
Beware that the 1999 version of OCD Commander silently truncates S-records longer than 20 bytes during download, so you'll need to patch objcopy or write your own S-record generator to work around this.
They have flash programming software available for download, but the website doesn't mention that it isn't actually free: you have to pay for the key to use it.
If you're prepared to do your debugging under Windoze, you can use the Cygwin tools from http://sources.redhat.com/cygwin/ to build a gdb cross-debugger which runs under Windoze and uses the Wigglers.dll to talk to the target. Note that Wigglers.dll also loads other DLLs, so copy all the DLLs that come with OCD Commander from http://www.macraigor.com/ into your GDB directory and use the command:
(gdb) target ocd wiggler
You can also do this remotely, by using rproxy, described below.
They've ported their VisionXD UNIX debugger to Linux and it is available with their parallel port of Ethernet based probe. It can program flash parts and do source level debugging. It costs $6k or $8k depending on parport or Ethernet connection. The Windoze version is only $4k. :-/
Beware that the EST tools don't handle the Linux zImage properly. See: http://lists.linuxppc.org/listarcs/linuxppc-embedded/200002/msg00073.html
There are many other sources of BDM hardware probes and debuggers, but many vendors still lag way behind as far as Linux support. Commercial solutions which do not support a Linux development host:
http://www.mentor.com/embedded/xray/index.html
http://www.tasking.com/products/PPC/ppcsolution.htm#debugger
http://www.sdsi.com/product/powerpc/index.php3#power
http://www.amc.com/products/powertap.html
Virtually all boards use a serial console on SMC1 for boot messages and general debugging. Connect it to a serial port on your Linux development machine, where you can run minicom to interact with the board.
Once the kernel is running, you can use gdb in several different ways to debug user space programs:
You can run gdbserver on your target and run gdb back on your development machine, even if you're cross developing. This requires far less resources than running all of gdb on your target. See: http://qslinux.org/docs/cross/gdb/index.html
If you're cross-developing, remember to configure your gdb as described earlier.
http://www.std.com/qqi/labslave/rproxy.html
This in an extended/enhanced gdbserver, which can also run on Windoze and talk to BDM devices not yet supported by Linux.
If you have lots of RAM, you can run gdb directly on your target. If you are cross developing, you need to configure gdb with:
% configure --host=powerpc-linux
Some kernels include "kgdb" support for using gdb for kernel debugging, enabled by configuring with CONFIG_KGDB.
You get these whenever something truly bad happens in the kernel. Learn to know and respect them -- they are your friend, not your enemy. For general info on how to understand them, see the file Documentation/oops-tracing.txt in the kernel source tree.
You'll get a long way just by looking up the instruction at the address indicated by NIP on the first line of the Oops in the output of:
objdump --disassemble vmlinuxThis will show you the instruction causing the fault. Work backwards to find the line of C source code associated with it, and add printk's around it to find what is going wrong.
For more help with decoding kernel panic messages, see: http://lists.linuxppc.org/listarcs/linuxppc-embedded/199912/msg00090.html
printk is an indispensible tool. You can use it to add checkpointing, print kernel values that you can't get to via /proc, etc etc. It can be called anywhere, including interrupt routines, provided you're prepared for some interesting output.
Note that during the boot process, the kernel "prints" lots of stuff, and it all goes into a buffer, to emerge quite late in the boot process when the serial console port is initialized with the call to console_init. This eventually calls register_console which will dump out the logged messages. So you can't necessarily assume that the kernel didn't get to your checkpoint just because the printk message didn't appear on the serial port during this part of the boot sequence.