Questions tagged [perf]

perf is Linux profiling tool for both Linux kernel and underlying hardware (hardware counters).

Filter by
Sorted by
Tagged with
2 votes
0 answers
35 views

A simple memcpy loop with stride of 8-16-32 Bytes, no L1 cache misses, what stalls backend cycles?

I am trying to understand the CPU cache performance in a single producer single consumer queue algorithm, but cannot pinpoint the cause of performance degradation in some cases. The following ...
xealits's user avatar
  • 2,143
0 votes
1 answer
62 views

Which tools for micro-benchmarking?

I'm not sure which tool to use for micro-benchmarking a C program. I would like to measure both: Memory usage, RSS ( Resident Set Size ) CPU cycles I did use perf record -g and perf script piped ...
jinTgreater's user avatar
1 vote
0 answers
20 views

What is futex-default-S?

I can't find more info about futex-default-S Is this a system call? module? or else? https://github.com/torvalds/linux/blob/06dc10eae55b5ceabfef287a7e5f16ceea204aa0/tools/perf/Documentation/perf-lock....
Mark K's user avatar
  • 865
1 vote
0 answers
55 views

perf instruction count

so i've been playing with perf and assembly i have the following program: .intel_syntax noprefix .global _start _start: mov cl, 2 mov ebx, 0b101 shr ebx, cl and bl, 1 je do_stuff ...
Joao Luca's user avatar
0 votes
1 answer
252 views

Unable to find any information about TLB on my computer or obtain information about hardware counters about TLB

The Ubuntu version I am using is Ubuntu 18.04.6 LTS, and the kernel version is 5.4.0-148 generic. My processor is 12th Gen Intel (R) Core (TM) i7-12700. I would like to know the number of TLB ...
citrusyi's user avatar
0 votes
0 answers
71 views

Security of perf under root access

when I use perf it asked for more permission to be used: You may not have permission to collect stats. I was lazy so I did sudo perf (some short program I created myself) but I later realize you can ...
NewbMaster66's user avatar
1 vote
0 answers
363 views

PAPI doesn't have available events on Ubuntu even though perf seems to be working properly

I'm trying to install PAPI on my machine: $ uname -a Linux XXXXXX 5.19.0-35-generic #36~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 17 15:17:25 UTC 2 x86_64 x86_64 x86_64 GNU/Linux and it does not ...
SkullKiddo's user avatar
3 votes
0 answers
203 views

Lower system call perfermance on aarch64 vs. x86-64

Does anyone know why system calls take longer to execute on 64-bit ARM (aarch64) compared to 64-bit Intel CPUs (x86-64)? Please see below my benchmarking setup. 64-bit ARM CPU: Ampere Altra Q80-30, ...
Dumi Loghin's user avatar
0 votes
0 answers
768 views

Unable to use perf to find L3 Cache miss rate

I am trying to get the L3 cache miss rate when I try to use perf. The following is the command I run: perf stat -e instructions,LLC-load-misses,LLC-loads ./program <arg1> <arg2> <arg3&...
darkdante's user avatar
1 vote
0 answers
216 views

How to measure memory bandwidth of an app?

My coworkers are trying to figure out a performance problem. I suspect they're reading lots of unnecessary memory and hitting a memory bandwidth problem but I don't want to suggest that unless I know ...
Ned's user avatar
  • 11
0 votes
0 answers
82 views

perf in docker doesn't show functions

I am trying to use perf on an application running in docker. In order to get a trace, I am running perf on the host and attaching to the application running in the container. This works fine. However, ...
jjcasmar's user avatar
  • 313
1 vote
0 answers
140 views

perf lock <...> for non-root user?

root can run perf lock record, whereas non-root can't: $ /code/linux/tools/perf/perf lock record true tracepoint lock:lock_acquire is not enabled. Are CONFIG_LOCKDEP and CONFIG_LOCK_STAT enabled? ...
Motiejus Jakštys's user avatar
1 vote
1 answer
692 views

how to use metrics with perf stat

perf list shows a bunch of what it calls metrics. The list starts off with List of pre-defined events (to be used in -e): Metrics: BAClear_Cost [Average Branch Address Clear Cost (fraction ...
rkg1's user avatar
  • 15
0 votes
1 answer
127 views

performance comparison with perf-stat

please check the below code. the code be implemented with two solutions. one is array, the other one is linking list. i expected the array version should be faster than the linking list one, because ...
chi-bin chou's user avatar
0 votes
1 answer
140 views

the column values shown by perf sched record don't make sense

The problem I'm seeing occurs with virtually any command I run with perf sched record I'll use ls as a very simple example. So if I do perf sched record ls followed by perf sched latency the ...
rkg1's user avatar
  • 15
0 votes
1 answer
312 views

what is the difference between itlb_misses.miss_causes_a_walk, itlb_misses.walk_completed, and itlb_misses.walk_pending

I ran perf stat -e itlb_misses.miss_causes_a_walk,itlb_misses.walk_completed,itlb_misses.walk_pending grep Hecuba /storage/nvme6/complete_shakespeare_4096_times and got Performance counter stats ...
rkg1's user avatar
  • 15
0 votes
1 answer
525 views

perf top not working as expected

I'm completely confused about the output of perf top I'm running a C program called sac2, which burns an entire CPU. Specifically it iterates through a 100 billion byte array adding all the bytes as ...
rkg1's user avatar
  • 15
1 vote
0 answers
606 views

How do I debug `perf`'s "Permission error mapping pages." when running `perf` on multiple processes in parallel?

perf sometimes gives me Permission error mapping pages. Consider increasing /proc/sys/kernel/perf_event_mlock_kb, or try again with a smaller value of -m/--mmap_pages. (current value: 4294967295,0) ...
Jason Gross's user avatar
2 votes
1 answer
1k views

linux-tools-common when on a newer kernel version

I'm currently running kernel 5.16.15-051615-generic - I've landed on it while diagnosing and trying to fix an audio issue. Everything works great now so I'm reluctant to go back to 5.4.0 I'd like to ...
CakePlusPlus's user avatar
2 votes
1 answer
988 views

How do I use perf trace record?

I can use perf trace as a low-overhead replacement for strace, e.g. to trace all Apache instances: perf trace -p $(pidof apache2 | tr ' ' ',') To run the trace only for up to 10 seconds: perf trace -...
Martin von Wittich's user avatar
3 votes
1 answer
1k views

how to set capabilities (setcap) on perf

I'd like to use the perf utility. I was following instructions to set up a privileged group of users who are permitted to execute performance monitoring and observability without limits (as instructed ...
levente.nas's user avatar
0 votes
0 answers
269 views

perf mem record <command> does not work as non root user

As a non root user, perf mem record does not work even though I have set /proc/sys/kernel/perf_event_paranoid to -1 and set CAP_SYS_ADMIN on /usr/bin/perf getcap /usr/bin/perf /usr/bin/perf = ...
Mike D's user avatar
  • 11
2 votes
0 answers
129 views

Error running python script

I am following this tutorial: How to analyze your system with perf and Python. While running the following perf script --gen-script python command I am getting an error. Python scripting not supported....
I192100 Mayra Ahmad's user avatar
1 vote
0 answers
116 views

linux perf: limiting to just some small periods of time

I'm writing a video game, and have used perf(1) and flame graphs to get some decent "whole program performance" information. However, I also want to investigate specific worst-case periods, ...
jwd's user avatar
  • 1,447
9 votes
1 answer
7k views

How to run linux perf without root

I want to benchmark an application of mine. Up to now I used gnu time, but perf yields much better stats. As a matter of principle I would like to go the route of a decicated perf user instead of ...
Eike's user avatar
  • 201
4 votes
1 answer
6k views

How does the linux perf tool get the miss rate of the l2 cache?

I want to use perf tool to get the miss rate of l2 cache. I know how to get L1, and LLC. perf stat -e L1-dcache-loads,L1-dcache-load-misses,L1-dcache-stores command perf stat -e LLC-loads,LLC-load-...
user avatar
1 vote
0 answers
372 views

How to take stack samples using `perf` based on wall-clock time

I am trying to use Linux's perf_events framework to investigate an issue with an application on one of our servers. Based on my reading about the perf tool, collecting stacks is relatively ...
zac's user avatar
  • 111
2 votes
2 answers
4k views

Perf permission for users other than root

I don't know why user is not able to collect perf stat for a service (nginx process which has been started by sudo). As you can see the kernel and perf versions match. $ uname -r 4.19.125 $ which perf ...
mahmood's user avatar
  • 1,191
0 votes
1 answer
240 views

Locating printf object/module

I would like to know which kernel module is called when printf is executed in my C code. How can I find that? Let's say, when I use ldd I can find which libraries are attached to my binary file. I am ...
mahmood's user avatar
  • 1,191
1 vote
0 answers
828 views

Can't install Perf on Kali Linux

I tried different ways to install perf but they are all failed: root@kali:~/Desktop# perf /usr/bin/perf: line 13: exec: perf_5.3: not found E: linux-perf-5.3 is not installed. root@kali:~/Desktop# ...
CSDUG's user avatar
  • 111
1 vote
0 answers
272 views

Where does `perf script` timestamps come from?

The third column from perf script seems to be close but not quite the uptime, where is that timestamp coming from? Is there a way to access that timestamp other than accessing sampled events? $ perf ...
Frederik Deweerdt's user avatar
0 votes
0 answers
552 views

High-frequency performance counter sampling using perf record/report

I want to retrieve performance counter counts at a high frequency (i.e. 100-200Hz) using the perf tool (similar in functionality to https://github.com/RRZE-HPC/likwid/wiki/likwid-perfctr#the-timeline-...
Justin Borromeo's user avatar
1 vote
1 answer
91 views

How to get sleep time of a program

I am trying to collect some statistics for a script that I run on a daily bases. I would like to get the amount of time that is actually spent sleeping. Is there a way to just get the sum of all sleep ...
nghj's user avatar
  • 11
1 vote
0 answers
18 views

Which of two methodologies is the correct one getting perf stats on a multi-threaded process?

I'm investigating the performance of a multi-threaded database sever. There is a particular workload that runs in roughly 61 seconds on a particular machine. The pid of the database process was 79894 ...
Richard Gostanian's user avatar
1 vote
0 answers
487 views

Perf tool, used for profiling, crashes on heavy load

When I am using "perf record -F 99 -g -p -- sleep 300" during average load, I am getting the perf.data file without any problem. But when the load is high, it is crashing. The only way to get an ok ...
kaevin's user avatar
  • 19
2 votes
0 answers
223 views

How to trace script execution events

My goal: To receive events from the kernel when a file/script is opened for execution with the absolute path of the file (or maybe the working directory, to resolve the absolute path of the file). I ...
opera127's user avatar
0 votes
1 answer
1k views

What does `perf stat` option `--cpu` is actually showing?

I am trying to understand the --cpu option of perf-stat. In the man pages it says: Count only on the list of CPUs provided. Multiple CPUs can be provided as a comma-separated list with no space: ...
adamm's user avatar
  • 101
12 votes
1 answer
10k views

Security implications of changing “perf_event_paranoid”

I'd like to use the perf utility to gather measurements for my program. It runs on a shared cluster machine with Debian 9 where by default the /proc/sys/kernel/perf_event_paranoid is set to 3, ...
Martin Ueding's user avatar
7 votes
1 answer
8k views

How long time does a context switch take in Linux (ubuntu 18.04)

I'm interessted in getting the number of context switches a two processes in a KVM vm takes on a singel CPU over some time. Earlier i have used perf, is this best practice? And how much time is used ...
emilrn's user avatar
  • 113
1 vote
0 answers
121 views

Determine average and maximum execution time of a task by using ftrace/perf

I've gathered some traces containing scheduling events ('sched:*') by using ftrace and perf record. By analyzing the results many of my questions could already be answered. However, I was wondering ...
Iniesta8's user avatar
  • 121
4 votes
0 answers
1k views

Does `perf trace -p <PID>` trace all threads?

strace -f -p <PID> traces system calls in all threads of a multi-threaded process. (This also follows forks, there is an option -b execve that tries to address that). Does perf trace -p <...
sourcejedi's user avatar
  • 49.9k
2 votes
1 answer
471 views

Is there a command to show tracepoints "live", which includes the PID?

I have tried looking at the documentation for perf script, perf trace, and trace-cmd, including the list of commands in "SEE ALSO". I can trace e.g. sched:sched_process_exec "live" using perf trace -...
sourcejedi's user avatar
  • 49.9k
3 votes
0 answers
750 views

Perf Tool monitoring by PID

I'm using linux perf tool to monitor the performance of some processes. The documentation tells that you can flag it to collect profiles on per-thread, per-process and per-cpu basis. I get that the ...
Nikhilesh Singh's user avatar
1 vote
1 answer
3k views

Why 'perf' needs to match the exact running Linux kernel version?

If you try to run a random perf binary that does not match your currently running Linux kernel, it says: $ perf WARNING: perf not found for kernel 4.13.0-45 Of course, if I get the perf for this ...
LeandroN.'s user avatar
  • 113
0 votes
0 answers
2k views

unresolved symbols sudo perf record/report

I am trying to use perf to profile a program that I have written on a work server. I have sudo privilege but should limit what I do with it. $ rm perf.data $ perf record ./catch-hedging ...
Bowie Owens's user avatar
3 votes
1 answer
2k views

Why does `perf stat` show 0 context switches?

I ran a shell pipeline under perf stat, using taskset 0x1 to pin the whole pipeline to a single CPU. I know taskset 0x1 had an effect, because it more than doubled the throughput of the pipeline. ...
sourcejedi's user avatar
  • 49.9k
2 votes
1 answer
3k views

Why does `perf stat -a` show clock (Ghz) lower than my cpu is rated?

Why does perf stat -a show a clock speed three times lower than my CPU is rated for? I don't think power management is an issue, because I made sure the test ran for a whole second to allow the cpu ...
sourcejedi's user avatar
  • 49.9k
1 vote
1 answer
1k views

Perf Error While Loading Shared Libraries

I was on CentOS 7.3 on x86_64 using perf compiled on the system itself from Kernel 4.13.7 source. It worked without any issue and was able to profile whichever application I wanted to. For some ...
Chetan Arvind Patil's user avatar
1 vote
0 answers
221 views

How could I add support for an event modifier in perf

I need to account power for the kernel space, but unfortunately the k modifier are not supported for the power/energy-cores/. I have an example of what I'm trying to do below. I'm reading the source ...
Leandro Silva's user avatar
2 votes
1 answer
3k views

How to show the report of Linux perf record -g without call graph data?

I collected a performance profile of an application with perf record -g. By default, perf report displays call chain data: 100.00% 0.00% htop htop [.] main | ...
Alexander Pozdneev's user avatar