I want to start a process and measure the cpu time (user+sys) it needs until it terminates.

I know I can use the wait4 system call, which returns a struct with user and system time (I use the sum of both times).

I can also use the cpuacct cgroup subsystem, put the process into a new cgroup, and read the cpuacct.usage file (which contains the combined user and system time).

I know the former has problems when the process spawns subprocesses (sometimes their time does not get counted). However, I thought that for a single-process case the two measurements should be almost equal. I expected the cpuacct value to be a little bit smaller, because I can only put the process into the cgroup after it has been created (but I do this immediately after fork, so the difference should be minimal).

Now for processes that take a few seconds, I see the expected results.


wait4:   8.292518s
cpuacct: 8.299105444s

wait4:   13.788861s
cpuacct: 13.796484557s

wait4:   24.229514s
cpuacct: 24.234132965s

wait4:   84.101255s
cpuacct: 84.104336222s

However, for processes that take longer, I get a different result:

wait4:   155.309706s
cpuacct: 155.306291274s

wait4:   505.547594s
cpuacct: 505.526723631s

wait4:   897.180069s
cpuacct: 897.131232685s

Now the cpuacct value is a little bit lower, which I would never expect to happen. Does somebody have an idea why this happens?

I do not care about 0.1s accuracy, and I have never experienced a higher difference. Can I be confident, that the difference will always be that small? Or are there possible scenarios where the difference will be larger?

The cpuacct documentation states that the value may be imprecise in 2 cases. However, both cases do not apply here, as I am on a 64bit machine, and I can wait an arbitrary amount of time after the process finishes, the reading of cpuacct.usage will stay constant (so I strongly believe it is not an outdated value).

Precise setup:
My starting process is a Python process. I use subprocess.Popen() to spawn the task and os.wait4() to get the time. For using the cgroups I pass a function to the preexec_fn parameter of Popen that adds the new process to the cgroup (so this is done between the fork and exec system calls). The started process is a Java VM. I don't think this is relevant, but I wanted to provide this information just in case.

  • They look the same to me... really too near to be true. Did you run your timings on an otherwise (almost) idle machine? The differences are of the order of 0.1%, which is excellent agreement. Note that the "system" time accounts also for (smallish) timeslices when the CPU is being charged to you, but the kernel is e.g. servicing an interrupt due to something else. – vonbrand Jan 25 '13 at 12:58
  • @vonbrand Thank's for you thoughts. Yes, the machine is otherwise idle. Interesting to hear about interrupt servicing time. Shouldn't this be the same for both measurements? No idea why the time difference consistently reverses for larger measurements? – Philipp Wendler Jan 26 '13 at 14:34
  • Do you have a source for "I know the former has problems when the process spawns subprocesses (sometimes their time does not get counted)"? I've been looking into a bug that seems related to this. – Steve Aug 24 '15 at 21:34
  • @SteveN I don't have a reference for this, but you can easily verify it yourself. The CPU time of a grand-child is only included in the CPU time of a child if the grand-child terminated before the child and the child has waited for the termination of the grand-child with one of the wait system calls. Otherwise the CPU time of the grand child is not yet included or even completely lost. – Philipp Wendler Aug 25 '15 at 21:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.