When debugging an executable that uses a library generated with libtool1, you might encounter the following error:

$ gdb ./tests-mylib
"tests-mylib": not in executable format: File format not recognized
  • libmylib.so: is a dynamic library generated with libtool.
  • tests-mylib: is an executable that uses the mylib library.

For those thinking I forgot to add the -g compilation option, this error occurs even when using -g.

The error is due to the fact that we are trying to execute a script generated by libtool, which is a wrapper over the actual program to facilitate its execution2.

To debug our test program tests-mylib

libtool --mode=execute gdb tests-mylib
1

Tool that belongs to the Autotools used to create portable software libraries.

2

More information in libtool documentation.