I want to recursively delete all files in directories and subdirectories with number of lines less than 10, and am currently using the following command

find . -type f -name "*.txt" | while read; do     
(($(cat $THISFILE | wc -l) < 10)) && rm -vf "$THISFILE"; done 

I have already used find with xargs to parallelize some commands, but here, I don't know how to do it because of the test on lines with wc -l

How could I make go faster with (or without !) xargs ?


find itself can not execute in parallel (not that I know of).

xargs can do it, and the simplest way to do it with xargs is to wrap it in a shell script.

But before that you should optimize your condition itself. cat is useless unless actually concatenating files. And you don't need to count ALL lines just to determine that a file has 10 or more. So I suggest a condition such as this:

[ $(head -n 10 "$file" | wc -l) -lt 10 ] && echo rm "$file"

which reads only the first 10 lines at most and deletes the file if it doesn't have that many (rm is a bit dangerous, so I added echo so you can test it first). Unlike cat, head actually stops reading after reaching 10 lines, so if you have files with lots more than 10 lines in your directory this should speed the process up a lot.

Wrapped in a shell script like so:


for file in "$@"
    [ $(head -n 10 "$file" | wc -l) -lt 10 ] && echo rm "$file"

You can use find + xargs for multi processing:

find . -type f -name "*.txt" -print0 | xargs -0 -P 4 -n 8 ./rm10lines.sh

The -P 4 (four processes) and -n 8 (8 arguments per call to the shell script) are examples, tune it to your liking. Use a larger -n if you know that you have a lot of files to reduce the overhead by respawning your shell script.

| improve this answer | |
  • looks like what I want :) I appreciate especially the head -n 10 tip – gdz Feb 5 '13 at 14:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.