Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I want to recursively delete all files in directories and subdirectories with number of lines less than 10, and am currently using the following command

find . -type f -name "*.txt" | while read; do     
(($(cat $THISFILE | wc -l) < 10)) && rm -vf "$THISFILE"; done 

I have already used find with xargs to parallelize some commands, but here, I don't know how to do it because of the test on lines with wc -l

How could I make go faster with (or without !) xargs ?

share|improve this question

1 Answer 1

up vote 3 down vote accepted

find itself can not execute in parallel (not that I know of).

xargs can do it, and the simplest way to do it with xargs is to wrap it in a shell script.

But before that you should optimize your condition itself. cat is useless unless actually concatenating files. And you don't need to count ALL lines just to determine that a file has 10 or more. So I suggest a condition such as this:

[ $(head -n 10 "$file" | wc -l) -lt 10 ] && echo rm "$file"

which reads only the first 10 lines at most and deletes the file if it doesn't have that many (rm is a bit dangerous, so I added echo so you can test it first). Unlike cat, head actually stops reading after reaching 10 lines, so if you have files with lots more than 10 lines in your directory this should speed the process up a lot.

Wrapped in a shell script like so:

#!/bin/bash

for file in "$@"
do
    [ $(head -n 10 "$file" | wc -l) -lt 10 ] && echo rm "$file"
done

You can use find + xargs for multi processing:

find . -type f -name "*.txt" -print0 | xargs -0 -P 4 -n 8 ./rm10lines.sh

The -P 4 (four processes) and -n 8 (8 arguments per call to the shell script) are examples, tune it to your liking. Use a larger -n if you know that you have a lot of files to reduce the overhead by respawning your shell script.

share|improve this answer
    
looks like what I want :) I appreciate especially the head -n 10 tip –  gdz Feb 5 '13 at 14:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.