I have a 12 cores server and I want to run a script which take a file as input argument and does processing on it. I want to use 8 core and no further to do parallel processing on 8 files at a time on 8 cores and then jump to next once the processing for 1 is complete.

I have tried doing it with xargs like so:

ls /data/paths/ | grep new | xargs -i -P 8 -n 1 bash main.sh {}

But when I see the core the CPU utilization is attached in the snapshot.enter image description here

I have also tried doing with parallel as follows:

ls /data/paths/ | grep new | parallel -j 8 --no-notice bash main.sh {}

My condition that I have to keep it limited to 8 CPUs.


When I run ps -ef | grep main.sh it shows 11-12 processes instead of 8.

  • 1
    Your CPU utilization doesn't tell us anything about how many processes are started by xargs or parallel. The scheduler may well assign a process to various cores in its lifetime.
    – muru
    Jun 4, 2019 at 6:48
  • @muru when I run ps -ef | grep main.sh it shows 11 processes instead of 8.
    – nainometer
    Jun 4, 2019 at 7:01
  • That way you're counting both grep, and xargs at the least in addition to your script's processes. Possibly something else as well.
    – muru
    Jun 4, 2019 at 7:03
  • 1
    Please show info in image also as text (it is currently inaccessible). Jun 4, 2019 at 7:59
  • 3
    Your utilisation is approximately 66.6% = ⅔ = 8 / 12 Therefore jobs is doing its job. However the OS, is balancing it. Jun 4, 2019 at 8:01

1 Answer 1


You can use taskset to set the cpu affinity of a process. The following will make sure that your xargs command only uses the first 8 virtual CPU cores:

ls /data/paths/ | grep new | taskset -c 0-7 xargs -i -P 8 -n 1 bash main.sh {}

Note that from man taskset:

A user must possess CAP_SYS_NICE to change the CPU affinity of a process.

  • 1
    This is also true for GNU Parallel (taskset will even make it only detect 8 cores, so -P is not needed): ls /data/paths/ | grep new | taskset -c 0-7 parallel bash main.sh
    – Ole Tange
    Jun 8, 2019 at 16:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .