中文 EN

4.6.3. Using GDB to Debug Applications

4.6.3.1. GDB Overview

GDB (GNU Debugger) is a powerful open-source debugging tool used to debug programs written in C, C++, Fortran, and other languages. It allows developers to pause program execution, inspect variable values, examine control flow, and fix bugs. GDB is open-source software widely used in Linux and other Unix-like operating systems, and is tightly integrated with the GCC compiler.

GDB is primarily used for debugging user-space programs. It was originally developed by Richard Stallman and The GNU Project to provide a powerful and free debugging tool for open-source software. Below is the development history of GDB:

• 1986: The first version of GDB was released. It was developed by Stallman in the early stages of his work, mainly to debug software from the GNU Project, especially programs compiled by GCC (GNU Compiler).

• 1990s: As open-source software and the GNU Project evolved, GDB became widely adopted in various development projects. It supported multiple programming languages (such as C, C++, Fortran), and gradually added support for more operating systems and platforms.

• 2000s: GDB introduced additional features such as multi-threaded debugging, remote debugging, and hardware-related debugging (e.g., using JTAG interfaces).

• 2010s to present: GDB continues to be actively maintained, with added support for modern hardware architectures (such as ARM, RISC-V), and further enhancements in remote and embedded system debugging. Its capabilities continue to expand, making GDB one of the most powerful debugging tools in the world.

4.6.3.2. Main Features of GDB

The main features of GDB include the following aspects:

• Program Execution Control

  • Allows starting and controlling program execution (step-by-step execution, skipping, pausing, etc.).

  • Supports setting breakpoints at different locations in the program (e.g., line numbers or functions).

• Variable Inspection

  • Enables checking and modifying variable values during program execution, helping developers track program state.

• Stack Tracing

  • When a program crashes or encounters an exception, GDB can display the call stack (backtrace), helping developers locate the source of the problem.

• Dynamic Debugging

  • Supports dynamically loaded libraries and functions, allowing debugging during program runtime.

• Source-Level Debugging

  • Allows debugging using source code (e.g., C or C++ source files), displaying variables, function calls, and execution positions.

• Multi-Platform Support

  • Supports multiple architectures (such as x86, ARM) and cross-platform debugging.

4.6.3.3. Common GDB Debugging Commands

A summary of commonly used GDB debugging commands is listed below:

Command	Description
l (list)	Displays context around the current line, showing 10 lines of code at a time. Line numbers or function names can be specified.
r (run)	Runs the program. Executes directly if no breakpoints are set; starts from the first breakpoint otherwise.
b (breakpoint) <line_number>	Sets a breakpoint at the specified line number.
b <source_file>:<function_name>	Sets a breakpoint at the first line of the specified function in the source file.
b <source_file>:<line_number>	Sets a breakpoint at the specified line in the source file.
info b	Displays information about current breakpoints, including hit counts.
d (delete) <breakpoint_number>	Deletes the specified breakpoint. Cannot use line numbers to delete. Breakpoint numbers increment continuously if GDB hasn't been exited.
d breakpoints	Deletes all breakpoints.
disable b (breakpoints)	Disables all breakpoints.
enable b (breakpoints)	Enables all breakpoints.
disable b (breakpoint) <number>	Disables the breakpoint with the specified number.
enable b (breakpoint) <number>	Enables the breakpoint with the specified number.
enable breakpoint	Enables a specified breakpoint, making it active.
n (next)	Executes line by line, skipping into function bodies.
s (step)	Steps into each statement, entering function bodies.
bt (backtrace)	Displays the current call stack, showing the sequence of function calls.
set var	Modifies the value of a variable.
p (print) <variable_name>	Prints the value of the specified variable.
display <variable_name>	Tracks and displays the value of a variable; shows its current value every time execution stops.
undisplay <variable_name>	Removes a previously set variable tracking.
until <line_number>	Jumps to the specified line, executes the code in between, and stops.
finish	Executes the current function until it returns, then stops at the function call site.
c (continue)	Continues execution from the current breakpoint until the next breakpoint is reached.

Note: The text in parentheses is the full name of the command.

For detailed GDB documentation, refer to the official website: GDB: The GNU Project Debugger.

4.6.3.4. Introduction to Specific GDB Debugging Methods

Writing a Test Program

Write an application on the PC and compile it with the -g option using cross-compilation to generate a debuggable binary:

Example demo.c:

#include <stdio.h>

int main()
{
    int a = 0;
    char i = 0;
    for(i =0; i <10; i++)
    {
        a = i + 1;
        int b = 10/a;
        printf("b = %d\n",b);
    }
    return 0;
}

Compilation

Note: During compilation, specify the toolchain used by the X5 BSP: opt/arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc. For convenience, it is recommended to create aliases in .bashrc:

alias arm_gcc='/opt/arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc'
alias arm_gdb='/opt/arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gdb'

The specific compilation process is as follows:

$ arm_gcc -g demo.c -o gdb_demo
$ ls
demo.c  gdb_demo
$ file gdb_demo
gdb_demo: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1,
for GNU/Linux 3.7.0, with debug_info, not stripped

This generates the required gdb_demo.

GDB Debugging Process

Transfer gdb_demo and demo.c to the target board (e.g., /userdata directory), then proceed with GDB debugging:

root@buildroot:/userdata# chmod +x gdb_demo
root@buildroot:/userdata# gdb gdb_demo
GNU gdb (GDB) 13.2
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-buildroot-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from gdb_demo...
# Display 10 lines of demo.c starting from line 1
(gdb) l 1
warning: Source file is more recent than executable.
1       #include <stdio.h>
2
3       int main()
4       {
5           int a = 0;
6           char i = 0;
7           for(i =0; i <10; i++)
8           {
9               a = i + 1;
10              int b = 10/a;
# Run the program until a breakpoint is encountered
(gdb) r
Starting program: /userdata/gdb_demo
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".
b = 10
b = 5
b = 3
b = 2
b = 2
b = 1
b = 1
b = 1
b = 1
b = 1
[Inferior 1 (process 2581) exited normally]
# Set a breakpoint at line 10 of demo.c
(gdb) b 10
Breakpoint 1 at 0x4005e8: file demo.c, line 10.
# View currently set breakpoints
(gdb) info b
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00000000004005e8 in main at demo.c:10
(gdb) r
Starting program: /userdata/gdb_demo
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".

Breakpoint 1, main () at demo.c:10
10              int b = 10/a;
# Monitor changes in variable 'b'
(gdb) display b
1: b = 65535
# Continue execution
(gdb) c
Continuing.
b = 10

Breakpoint 1, main () at demo.c:10
10              int b = 10/a;
1: b = 10
(gdb) c
Continuing.
b = 5

Breakpoint 1, main () at demo.c:10
10              int b = 10/a;
1: b = 5
(gdb) c
Continuing.
b = 3

Breakpoint 1, main () at demo.c:10
10              int b = 10/a;
1: b = 3
# Exit GDB debugging
(gdb) q

Analyzing Core Dump Files with GDB

Using GDB, we can effectively analyze the state of a program at the time of a crash, quickly locate the issue, and fix it.

Write a test case to simulate a crash:

#include <stdio.h>

int main() {

    printf("Program will now crash due to null pointer dereferencing.\n");

    // Create a null pointer
    int *ptr = NULL;

    // Attempt to dereference the null pointer
    *ptr = 10;  // Dereferencing a null pointer causes a crash

    return 0;
}

Compile:

arm_gcc -g crash_example.c -o crash_example

Here, arm_gcc is an alias for the compiler tool created in .bashrc:

# ~/.bashrc
alias arm_gcc='/opt/arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc'

Transfer the compiled binary to the target board and run the test:

1. Execute the file to trigger the crash

   root@buildroot:/userdata# ./crash_example
   Program will now crash due to null pointer dereferencing.
   Segmentation fault (core dumped)
   

2. Retrieve the generated Core dump file

   On X5 devices, Core dump files are stored in /userdata/log/coredump. Copy it to /userdata for analysis:

   root@buildroot:/userdata/log/coredump# ls
   core-crash_example-2685-36246
   root@buildroot:/userdata/log/coredump# cp core-crash_example-2685-36246 ../../
   

3. Run GDB to analyze the Core dump file

   # Start GDB analysis
   root@buildroot:/userdata# gdb crash_example core-crash_example-2685-36246
   GNU gdb (GDB) 13.2
   Copyright (C) 2023 Free Software Foundation, Inc.
   License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
   This is free software: you are free to change and redistribute it.
   There is NO WARRANTY, to the extent permitted by law.
   Type "show copying" and "show warranty" for details.
   This GDB was configured as "aarch64-buildroot-linux-gnu".
   Type "show configuration" for configuration details.
   For bug reporting instructions, please see:
   <https://www.gnu.org/software/gdb/bugs/>.
   Find the GDB manual and other documentation resources online at:
       <http://www.gnu.org/software/gdb/documentation/>.
   
   For help, type "help".
   Type "apropos word" to search for commands related to "word"...
   Reading symbols from crash_example...
   [New LWP 2685]
   [Thread debugging using libthread_db enabled]
   Using host libthread_db library "/lib/libthread_db.so.1".
   Core was generated by `./crash_example'.
   Program terminated with signal SIGSEGV, Segmentation fault.
   #0  0x00000000004005e4 in main () at crash_example.c:11
   11          *ptr = 10;  // Dereferencing a null pointer causes a crash
   # Display surrounding code of the error line
   (gdb) l 10
   5           printf("Program will now crash due to null pointer dereferencing.\n");
   6
   7           // Create a null pointer
   8           int *ptr = NULL;
   9
   10          // Attempt to dereference the null pointer
   11          *ptr = 10;  // Dereferencing a null pointer causes a crash
   12
   13          return 0;
   14      }
   # Display the current call stack
   (gdb) bt
   #0  0x00000000004005e4 in main () at crash_example.c:11
   # Display values of all local variables
   (gdb) info locals
   ptr = 0x0
   

4.6.3.5. Common Issues

The following issues are commonly encountered during GDB debugging:

Issue	Description	Possible Causes	Solutions
Unable to start GDB or connect to target program	GDB fails to start or cannot connect to the process being debugged	Mismatch between GDB and target architecture, insufficient permissions, missing symbol information in the target program	Ensure GDB matches the target architecture, run GDB with sudo, ensure -g is used during compilation to generate debug symbols
Missing debug symbols	GDB reports missing symbols, preventing variable or stack inspection	Debug symbols not included during compilation, excessive optimization preventing source-to-machine code mapping	Compile with -g to generate debug symbols, use lower optimization levels (e.g., -O0)
GDB fails to load target program symbols	Debugger cannot load program symbols, preventing variable or stack inspection	GDB cannot locate the correct symbol file, or the symbol file version does not match the program	Manually load symbol file, use set solib-search-path to specify path to shared library symbol files
Incorrect thread switching in multi-threaded debugging	GDB fails to identify or switch threads correctly in multi-threaded programs	Synchronization issues between GDB and multi-threaded program, multi-threading debugging not enabled, missing thread library linkage	Use info threads to view thread info, thread <id> to switch threads, ensure multi-threading support is enabled at compile time
Breakpoints not triggered or skipped	Set breakpoints are ineffective or skipped during execution	Invalid breakpoint location, optimized code skips target line, loss of dynamic library symbol information	Use info breakpoints to check breakpoint status, recompile with lower optimization, ensure shared library symbols are loaded
Issues debugging kernel or bare-metal programs	Connection problems or inability to load symbols when debugging kernel or bare-metal programs	Missing debug symbols in kernel or bare-metal program, unstable GDB connection to target in bare-metal environment	Enable debug symbols during compilation, use appropriate remote debugging methods (e.g., serial, JTAG) to connect to target
GDB and target program version mismatch	GDB reports symbol file mismatch or cannot find symbols	Symbol file used by GDB does not match the version of the target program	Ensure GDB uses the correct symbol file matching the target program version
Network connection issues in remote debugging	Unstable or failed connection when using GDB for remote debugging	Network configuration errors, firewall blocking debug port, GDB server not properly started on target	Check network settings, ensure firewall does not block debug port, confirm GDBserver is running and listening on correct port
Memory leaks or data inconsistency	Memory leaks or data inconsistency errors occur during debugging	Uninitialized variables, out-of-bounds access in code, variable tracking lost due to high optimization or missing debug symbols	Use valgrind to detect memory leaks, compile with debug options, avoid high optimization levels

4.6.3.6. References

GDB: The GNU Project Debugger
gdb Debugging Full Example (Tutorial): ncurses