0

Assume, someone searches for files that match some criteria using the find command and pipes the result into the rm command.

find / -type f -name "*.jpg" | xargs rm 

First, I am not sure if the command above will get an error if the number of found files is very big, because I guess there is a limit to the number of arguments a command like rm can hold.

Than, I know that I can set how many files are passed as arguments at once to rm. Xargs with -n3 would, for instance, pass three arguments to rm execute the command and repeat it until all files have been passed as arguments.

Now does it make somehow a difference what value I take for the -n option of xargs ? Do these command lines differ in some aspect , like for instance the duration of execution:

find / -type f -name "*.jpg" | xargs rm 

find / -type f -name "*.jpg" | xargs -n1 rm 

find / -type f -name "*.jpg" | xargs -n2 rm 

find / -type f -name "*.jpg" | xargs -n3 rm

.
.
.

find / -type f -name "*.jpg" | xargs -n999 rm  
1

The limitation you mentioned is generally about the exec buffer used, not specific to individual commands. The purpose of xargs is exactly to address that problem; xargs will take as many arguments as possible to feed the command. This will get you the least command calls and thus good performance. Reducing the amount of arguments for the command by explicitly specifying -n will not gain anything in this respect. You can use -n e.g. in cases where the commands expect a specific number of arguments, with the special case that the command can be called once for each argument. (Note that findhas also an option -exec and a special terminator + to facilitate a similar behaviour.)

  • what is the exec buffer ? – Abdul Al Hazred Mar 24 '15 at 21:22
  • It refers to the exec() system call, part of the (fork/exec) mechanism to create processes in Unix. It's a fixed size buffer where arguments are passes. (It's nothing a user could influence, but that there's a limitation is good to known for Unix users.) – Janis Mar 24 '15 at 21:29
0

Since I asked a similar question: It made a difference when I called md5sum recursively:

time find -type f -print0 | ...
... xargs -0 --max-procs 0 md5sum  37,98s user 2,45s system 99% cpu 40,464 total
... xargs -0 -n100 --max-procs 0 md5sum  41,18s user 2,66s system 327% cpu 13,374 total
... xargs -0 -n10 --max-procs 0 md5sum  45,20s user 2,93s system 573% cpu 8,396 total
... xargs -0 -n1 --max-procs 0 md5sum  45,90s user 3,01s system 591% cpu 8,263 total

find -type f -exec md5sum {} \;  37,87s user 2,62s system 99% cpu 40,596 total

As you can see the -n parameter is the only thing that brought me some speed up

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.