Skip to content

Debugging & Profiling Tips

Tomas Mlcoch edited this page Mar 21, 2014 · 16 revisions

Tools

  • Valgrind - Holy grail of debugging :)
  • Valgrind Massif - Heap profiler
  • GooglePerformanceTools (gperftools) - Fast, multi-threaded malloc() and nifty performance analysis tools
  • Qt Creator - Great IDE, with great debugging capabilities (it uses GDB), it provide also stuff like Valgrind memchecking and profiling. And whole this functionality is available in pretty GUI!

Additional compilation options

For example for gperftools use:

cmake -DCMAKE_BUILD_TYPE:STRING=DEBUG -DCMAKE_C_FLAGS:STRING=" -ltcmalloc " ..

Sometimes, mostly when you are debugging third-party software, you need to modify the original compiler flags that were set by autotools or similar build system. Then you can try:

make CFLAGS="-g -Og"

or set the flags right to the ./configure script by:

CPPFLAGS='-Og' ./configure   # For programs compiled by a C++ compiler

or:

CFLAGS='-Og' ./configure   # For programs compiled by a C compiler

Quick howto

Valgrind

valgrind prog

If the program uses Glib2 library, there can be a lot of false positives. So use the command instead:

G_SLICE=always-malloc G_DEBUG=gc-friendly valgrind prog

Tip: Add this line to your ~/.bashrc: alias gvalgrind='G_SLICE=always-malloc G_DEBUG=gc-friendly valgrind' and use gvalgrind instead of valgrind.

Useful options

  • --leak-check=full - show details for each definitely lost or possibly lost block, including where it was allocated

Valgrind - Massif

A heap profiler

valgrind --tool=massif prog
# And then
ms_print massif.out.12345

Massif useful options

  • --stacks=yes - Specifies whether stack profiling should be done.
  • --pages-as-heap=yes - By default massif measures only memory allocated by hi-level function like malloc, calloc, new, etc. System calls like mmap, mremap, and brk are not measured by default. This options enables this measurement. This will lead to measure all memory in process.

Useful options for ms_print

  • --threshold=50.0 - Allocation tree entries that account for less than this will be aggregated. (e.g. ms_print --threshold=50.0 massif.out.12345)

gperftools

HEAPCHECK=normal prog           # Heap checker
HEAPPROFILE=/tmp/netheap prog   # Heap profiler
CPUPROFILE=/tmp/profile prog    # CPU profiler

Note: If it complain that PPROF_PATH is not set and memory addresses are not translated to symbols properly, prepend PPROF_PATH=/usr/bin/pprof (of course with path that match your system - use whereis pprof to obtain the path)

Analyzing gperftools outputs

pprof --text prog /tmp/netheap.0001.heap 

Another debug tools/techniques available on any Linux system

Top

You can run your program prog and then set watch its memory run-time CPU and memory usage.

top -d 0.2 -p $(ps -A | grep prog | awk '{print $1}')

Notes

  • VIRT -- Virtual Image (kb) The total amount of virtual memory used by the task. It includes all code, data and shared libraries plus pages that have been swapped out and pages that have been mapped but not used.
  • RES -- Resident size (kb) The non-swapped physical memory a task has used.
  • SHR -- Shared Mem size (kb) The amount of shared memory used by a task. It simply reflects memory that could be potentially shared with other processes.

Time

Simple resource usage. Without parameters it returns three time values:

  • real - Elapsed real time between invocation and termination.
  • user - User CPU time (the sum of the tms_utime and tms_cutime values in a struct tms as returned by times(2))
  • sys - System CPU time (the sum of the tms_stime and tms_cstime values in a struct tms as returned by times(2))
time prog

If you wanna see more information, set the TIME environ variable with formatting string.

For example:

TIME="\nMaximum resident set size of the process during its lifetime: %M Kbytes\nAverage resident set size of the process: %t\nAverage total (data+stack+text) memory use of the process: %K\nAverage size of the process's unshared data area: %D\nAverage size of the process's unshared stack space: %p\nAverage size of the process's shared text space: %X\n\n"

Ulimit

Provides control over the resources available to the shell and to processes started by it, on systems that allow such control.

ulimit -Sv 500000  # Set soft limit of virtual memory to 500Mb
prog

Note

-m option for set maximum resident size is not working on Linux

Useful links