1

I'm struggling getting perf_events to give me stack traces with symbols, despite reading many tutorials on the subject and doing (I think) all the necessary things. It's possible that my local install of perf (details on that below) is somehow botched? Anyway, here's what I did:

main.cpp is a simple C++ program that calls a few functions defined in the same file, allocates some memory and frees it, and prints a few things out.

compilation command:

gcc -std=c++11 -lstdc++ main.cpp -Og -fno-omit-frame-pointer -fno-inline -o arr_test

profile command:

perf record -a -g -- ./arr_test && perf report --stdio

I do get the following warnings about kernel symbols, but I don't think this should matter given that I only care about symbols in my application for now:

[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.052 MB perf.data (~2285 samples) ]
[kernel.kallsyms] with build id e22966849c48748782a1be4fe0ce94db6838b806 not found, continuing without symbols
[kernel.kallsyms] with build id e22966849c48748782a1be4fe0ce94db6838b806 not found, continuing without symbols
Warning:
Kernel address maps (/proc/{kallsyms,modules}) were restricted.

Check /proc/sys/kernel/kptr_restrict before running 'perf record'.

As no suitable kallsyms nor vmlinux was found, kernel samples
can't be resolved.

Samples in kernel modules can't be resolved as well.

Here's a snippet of the output:

# Overhead   Command      Shared Object
# ........  ........  .................
#
    83.27%  arr_test  arr_test         
            |          
            |--34.12%-- 0x400908
            |          0x7fe72b381ec5
            |          
            |--10.48%-- 0x400903
            |          0x7fe72b381ec5
            |          
            |--10.08%-- 0x4008b8
            |          0x7fe72b381ec5
            |          
            |--9.22%-- 0x4008e5
            |          0x7fe72b381ec5
            |          
            |--9.05%-- 0x4008da
            |          0x7fe72b381ec5
            |          
            |--8.49%-- 0x4008f0
            |          0x7fe72b381ec5
            |          
            |--6.87%-- 0x4008d5
            |          0x7fe72b381ec5
            |          
            |--6.23%-- 0x4008c2
            |          0x7fe72b381ec5
            |          
            |--4.76%-- 0x4008fd
            |          0x7fe72b381ec5
             --0.70%-- [...]

     8.02%  arr_test  [kernel.kallsyms]
            |          
            |--4.87%-- 0xffffffff81140b64
            |          0xffffffff81146646
            |          0xffffffff81182751
            |          0xffffffff811829eb
            |          0xffffffff8173317d
            |          0x7fe72bab86a7
            |          0x7fe72baa7e00

file info (shows "not stripped"):

$ file arr_test 
arr_test: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, not stripped

Details on my perf install (do any of these warnings prevent me from seeing symbols in stacks?)

Auto-detecting system features:
...                     backtrace: [ on  ]
...                         dwarf: [ OFF ]
...                fortify-source: [ on  ]
...                         glibc: [ on  ]
...                          gtk2: [ on  ]
...                  gtk2-infobar: [ on  ]
...                      libaudit: [ OFF ]
...                        libbfd: [ OFF ]
...                        libelf: [ OFF ]
...             libelf-getphdrnum: [ OFF ]
...                   libelf-mmap: [ OFF ]
...                       libnuma: [ on  ]
...                       libperl: [ on  ]
...                     libpython: [ on  ]
...             libpython-version: [ on  ]
...                      libslang: [ on  ]
...                     libunwind: [ OFF ]
...                       on-exit: [ on  ]
...                stackprotector: [ on  ]
...            stackprotector-all: [ on  ]
...                       timerfd: [ on  ]

config/Makefile:264: No libelf found, disables 'probe' tool, please install elfutils-libelf-devel/libelf-dev
config/Makefile:329: No libunwind found, disabling post unwind support. Please install libunwind-dev[el] >= 1.1
config/Makefile:354: No libaudit.h found, disables 'trace' tool, please install audit-libs-devel or libaudit-dev

How can I find my symbols in perf?

  • All, is there anything else I can add to help get this question answered? Thanks. – Kulluk007 Apr 19 '16 at 0:00
  • Did you ever find a solution for this ? I'm struggling with the same problem on x86-64. – Paul R Feb 23 '17 at 10:07
2

I'm compiling using more debugging options:

-Og -ggdb -g3 -fno-omit-frame-pointer

Then, when I record I'm not using -a option (that should monitor all the system processes), I'm using

perf record -e cycles -g --call-graph fp -- ./your_app your_args

Finally, to show the result I'm using

perf report -g graph

and the output looks like expected (note, I'm using debian 9 and the perf report output is ncurses based)

-   92.18%     0.00%  stsm     stsm                  [.] main                                                                                                ◆
   - main                                                                                                                                                    ▒
      - 91.77% STSM::run                                                                                                                                     ▒
         + 56.86% STSM::generateCandidates                                                                                                                   ▒
         - 25.22% STSM::detectBlocksOfAllSolidSequences                                                                                                      ▒
            + 23.42% STSM::detectSolidSequenceBlocksFromSolidSequence                                                                                        ▒
              0.81% Segment::unify                                                                                                                           ▒
         + 5.25% STSM::updateKernelsOfAllCandidates                                                                                                          ▒
           1.80% RangedSequence::range                                                                                                                       ▒
         + 1.45% STSM::updateMatchingPositions                                                                                                               ▒
           0.99% Segment::intersects                                                                                                                         ▒
+   92.18%     0.00%  stsm     libc-2.24.so          [.] __libc_start_main                                                                                   ▒
+   92.18%     0.00%  stsm     [unknown]             [k] 0x4d96258d4c544155                                                                                  ▒
+   91.77%     0.00%  stsm     stsm                  [.] STSM::run                                                                                           ▒
+   56.86%     6.74%  stsm     stsm                  [.] STSM::generateCandidates                                                                            ▒
+   49.99%    49.99%  stsm     stsm                  [.] Segment::intersects                                                                                 ▒
+   25.22%     0.00%  stsm     stsm                  [.] STSM::detectBlocksOfAllSolidSequences 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.