|Version 22 (modified by http://nedko.arnaudov.name/, 3 years ago)|
This page describes how to debug a D-Bus service. ladishd, jackdbus and jmcore are D-Bus services.
Table of Contents
There are three ways to get a backtrace of daemon crash with gdb:
- Running the daemon through gdb
- Attach gdb to already running daemon
- Post-mortem debugging via corefile
Under normal conditions D-Bus messagebus daemon starts the D-Bus service daemons, when they are needed. For the first method, in order to debug a D-Bus service daemon, its process must be started manually, from gdb. As a prerequisite, it must be ensured that the to-be-debugged daemon is not started. The required additional preparation steps are in the next chapter. They are not strictly needed for the two other methods.
If you are not sure what apps are accessing the dbus service it may be a good idea to disable autolaunching of the deamon by editing its .service file. D-Bus service files usually reside in /usr/share/dbus-1/services/ for session bus daemons and in /usr/share/dbus-1/system-services/ for system bus daemons. jackdbus, ladishd and jmcore are all session bus daemons:
[D-BUS Service] Name=org.jackaudio.service Exec=/usr/bin/jackdbus auto
[D-BUS Service] Name=org.ladish Exec=/usr/bin/ladishd
[D-BUS Service] Name=org.ladish.jmcore Exec=/usr/bin/jmcore
Renaming the file to something like org.xxx.service.back or changing the "Exec" line to invalid commandline will effectively disable autolanching.
Stop the service
If you know all the apps are accessing the service, you can just stop them. If no app requests access to the service, the D-Bus messagebus daemon will not start it.
For jackdbus these apps may be:
For ladishd these apps may be:
For jmcore the list is probably quite short:
Usually you want to stop qjackctl, laditools and gladish.
To quit a dbus service daemon, you can shut it down nicely:
- ladish_control exit
- jack_control exit
- dbus-send --session --print-reply=yes --dest=org.ladish.jmcore /org/ladish/jmcore org.ladish.jmcore.exit
or slaughter them:
- killall -9 jackdbus ladishd jmcore
Running the daemon through gdb
Now that daemons are down and you are sure they will not go up automatically. Debug them like this:
$ gdb <path_of_the_daemon_executable> ... gdb loads ... (gdb) set pagination 0 (gdb) run [<args>] ... daemon loads...
Typically the <path_of_the_daemon_executable> will be:
Of these three, only jackdbus needs an argument "auto". I.e. the gdb command is "r auto".
The contents of the service files specify the actual commandlines that are used by D-Bus messagebus daemon to start the service daemons.
Running gdb (attach mode)
Instead of running the daemon through gdb you can start gdb and attach to already running process. You have to run gdb as same user as the process to be debugged.
Where 17234 is the pid of the process to debug (as reported by ps for example).
Sometimes the attach fails and gdb quits. Sometimes it works after a retry.
Once you attach to the process, tell gdb to continue:
This is smarter way but it has a drawback - if you debug a daemon that relies on time critical stuff (jackdbus for example), it may get mad or act strange for a while. In case of jackdbus you may get a xrun. So once you attach, type 'c' quickly.
Reproduce the crash
Start apps that are required to reproduce the crash. This may include the apps that you have stopped during preparation (laditray, qjackctl, gladish).
Reproduce the crash.
This technique relies on the daemon coredump file. The advantage is that it gdb does not interact with the daemon process at all. This is especially useful for daemons like jackdbus that will produce xruns when gdb is accepting commands.
How and whether coredump files are generated is distro specific but often you can enable it like this:
ulimit -c unlimited
If you start an executable from this shell, a coredump should be generated once the executable crashes. Look for a file with .core suffix in the current directory.
You can then load the coredump file into gdb:
gdb <path_of_the_daemon_executable> <path_of_the_daemon_coredump>
Gather information about the crash
Once crash happens, gdb will notice it and will enter a command prompt mode. Often the most useful information is the backtrace of the crash:
For more complex bugs backtrace of all threads is important:
thread apply all bt
If you have renamed or tweaked the .service files, put them back to the original state to ensure normal operation of the LADI system. Start the apps that you stopped (laditray, qjackctl, gladish).