This page describes how to debug a D-Bus service. ladishd, jackdbus and jmcore are D-Bus services.

Intro

Under normal conditions D-Bus messagebus daemon starts the D-Bus service daemons, when they are needed. In order to debug a D-Bus service daemon, its process must be started manually, from gdb. As a prerequisite, it must be ensured that the to-be-debugged daemon is not started.

Preparing

Disable autolaunch

If you are not sure what apps are accessing the dbus service it may be a good idea to disable autolaunching of the deamon by editing its .service file. D-Bus service files usually reside in /usr/share/dbus-1/services/ for session bus daemons and in /usr/share/dbus-1/system-services/ for system bus daemons. jackdbus, ladishd and jmcore are all session bus daemons:

• /usr/share/dbus-1/services/org.jackaudio.service

  [D-BUS Service]
  Name=org.jackaudio.service
  Exec=/usr/bin/jackdbus auto
  

• /usr/share/dbus-1/services/org.ladish.service

  [D-BUS Service]
  Name=org.ladish
  Exec=/usr/bin/ladishd
  

• /usr/share/dbus-1/services/org.ladish.jmcore.service

  [D-BUS Service]
  Name=org.ladish.jmcore
  Exec=/usr/bin/jmcore
  

Renaming the file to something like org.xxx.service.back or changing the "Exec" line to invalid commandline will effectively disable autolanching.

Stop the service

If you know all the apps are accessing the service, you can just stop them. If no app requests access to the service, the D-Bus messagebus daemon will not start it.

For jackdbus these apps may be:

• laditools
• qjackctl
• ladishd
• gladish

For ladishd these apps may be:

• laditools
• gladish

For jmcore the list is probably quite short:

• ladishd

Usually you want to stop qjackctl, laditools and gladish.

To quit a dbus service daemon, you can shut it down nicely:

• ladish_control exit
• jack_control exit
• dbus-send --session --print-reply=yes --dest=org.ladish.jmcore /org/ladish/jmcore org.ladish.jmcore.exit

or slaughter them:

• killall -9 jackdbus ladishd jmcore

Debugging

There are three ways to get a backtrace of daemon crash with gdb:

• Running the daemon through gdb
• Attach gdb to already running daemon
• Post-mortem debugging via corefile

Running the daemon through gdb

Now that daemons are down and you are sure they will not go up automatically. Debug them like this:

 $ gdb <path_of_the_daemon_executable>
 ... gdb loads ...
 (gdb) set pagination 0
 (gdb) run [<args>]
 ... daemon loads...

Typically the <path_of_the_daemon_executable> will be:

• /usr/bin/jackdbus
• /usr/bin/ladishd
• /usr/bin/jmcore

Of these three, only jackdbus needs an argument "auto". I.e. the gdb command is "r auto".

The contents of the service files specify the actual commandlines that are used by D-Bus messagebus daemon to start the service daemons.

Running gdb (attach mode)

Instead of running the daemon through gdb you can start gdb and attach to already running process. You have to run gdb as same user as the process to be debugged.

attach 17234

Where 17234 is the pid of the process to debug (as reported by ps for example).

Sometimes the attach fails and gdb quits. Sometimes it works after a retry.

Once you attach to the process, tell gdb to continue:

c

This is smarter way but it has a drawback - if you debug a daemon that relies on time critical stuff (jackdbus for example), it may get mad or act strange for a while. In case of jackdbus you may get a xrun. So once you attach, type 'c' quickly.

Reproduce the crash

Start apps that are required to reproduce the crash. This may include the apps that you have stopped during preparation (laditray, qjackctl, gladish).

Reproduce the crash.

Post-mortem debugging

This technique relies on the daemon coredump file. The advantage is that it gdb does not interact with the daemon process at all. This is especially useful for daemons like jackdbus that will produce xruns when gdb is accepting commands.

How and whether coredump files are generated is distro specific but often you can enable it like this:

ulimit -c unlimited

If you start an executable from this shell, a coredump should be generated once the executable crashes. Look for a file with .core suffix in the current directory.

You can then load the coredump file into gdb:

gdb <path_of_the_daemon_executable> <path_of_the_daemon_coredump>

Gather information about the crash

Once crash happens, gdb will notice it and will enter a command prompt mode. Often the most useful information is the backtrace of the crash:

bt

For more complex bugs backtrace of all threads is important:

thread apply all bt

Cleanup

If you have renamed or tweaked the .service files, put them back to the original state to ensure normal operation of the LADI system. Start the apps that you stopped (laditray, qjackctl, gladish).