3

I have one process running on a RHEL 6.3 machine. Somehow this is taking a longer processing time than the same process running on another machine. How is it possible to know why this process is running slow or if any other parallel running process is slowing down this process?

  • 1
    Are you familiar with tools such as top? – slm Aug 19 '13 at 3:54
  • Yes.. But that only gives a list of processses and the memory allotted to each process right? Is it possible to find the answer to my question from top output? – user2349990 Aug 19 '13 at 3:56
2

You could try debugging what's going on by using either top or htop.

top & htop show the amount of resources a particular process is consuming. I would run your process like this: /usr/bin/time myproc to see how much time it's really taking to start. This command should show the same user time but different elapse times across the 2 machines.

Once you've established that the process is taking the same "effective" amount of time on the various systems you can then use either top or htop to determine what "other" processes are slowing the system down so that this process is taking longer on machineA vs. machineB.

Example

Say I have the following script, doit.bash.

#!/bin/bash

a=0
while [ 1 ]; do
  let a=a+1
  (( $a==100000 )) && exit
done

I then run it on unloaded system:

$ /usr/bin/time ./doit.bash 
1.25user 0.10system 0:01.38elapsed 98%CPU (0avgtext+0avgdata 1272maxresident)k
0inputs+0outputs (0major+366minor)pagefaults 0swaps

I then run this command to simulate some load on my box:

$ dd if=/dev/zero of=/dev/null

Repeating a run of doit.bash:

$ /usr/bin/time ./doit.bash 
2.32user 0.20system 0:02.98elapsed 84%CPU (0avgtext+0avgdata 1268maxresident)k
0inputs+0outputs (0major+365minor)pagefaults 0swaps

You can see that doit.bash took longer to run in both terms of the total user time and elapse time.

  • run1 - (user): 1.25 seconds - (elapse): 1.38 seconds
  • run2 - (user): 2.32 seconds - (elapse): 2.98 seconds

NOTE: make sure to Ctrl + C the running dd command when you're done.

0

Do you have any idea if the process is CPU or IO limited? If it needs only CPU, top is your friend. If it's IO limited, check out vmstat -Sm 5, iotop -od5 and iostat -m 5. Try to figure out if the problem is caused by slow poor device performance or if there's something else running on the worse performing system.

If you don't know if it's CPU or IO limited, start with top and look at the system IO wait value (wa) at the top. If one or more CPUs are waiting for IO, the IO wait value will raise. If you have lots of idle, and low IO wait value, then your process is waiting for some kind of syncronization events (probably some kind of locking either locally or over network).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.