Count result in a find/exec statement

Question

I have some Perl files which I have to modify through a simple sed call. Basically, I have to remove the first line of all of them. Currently, I managed to do this :

find <PATH> -type f -name "*.pl" -exec sed -i '1d' {} \;

However, I am using this line in a script and I would it to be a bit more... talkative. Thus, I decided to echo a counter (in real time), that would show how many files have been processed so far.

I know the count of Perl files can be retrieved through

PERL_FILE_COUNT=$(find <PATH> -name "*.pl" | wc -l)

Currently, I have this

remove_first_line()
{
    count=0
    echo -ne "Removing first line of Perl files ..."
    find <PATH> -type f -name "*.pl" -exec sed -i '1d' {} \; >/dev/null 2>&1

    if [ $? -eq 0 ]
        then echo "OK"
        then echo "FAILED"
    fi
}

Now, what I would like as an output, would be something like this:

"Removing first line of Perl files ... 1/209209"

And the value should be automatically updated. But I don't see how to increment the count variable with my find/exec statement. Basically, each time sed finished working a file, it should increment the count variable.

Answer 1

If you have bash 4, consider using globstar instead. It gives you recursive globbing.

shopt -s globstar
perlz=( **/*.pl ) # */ Hack to fix syntax highlighting
totes="${#perlz[@]}"
i=0
for file in "${perlz[@]}"; do
    printf 'Removing first line of Perl files … %d/%d\r' $((++i)) $totes
    ed -s "$file" <<< $'1d\nw' # You can use `sed` if you want to, but ed is an actual file editor
done
echo # print a final newline

This solution will work across files with crazy characters in their names and avoids a subshell.

But if bash 4 isn't an option, you can recreate this solution using find -exec +:

find . -name '*.pl' -exec bash -c 'totes=$#
  i=0
  for file; do
    printf "Removing first line of Perl files … %d/%d\r" $((++i)) $totes
    ed -s "$file" <<< $'\''1d\nw'\'' # Avoid these leaning toothpicks by putting this
                                     # script in a file.
  done
  echo # print a final newline
' bash {} +

However, this is subject to your system's ARG_MAX (unlike the above), so if the number of files is very large you could still end up with multiple runs over subsets of the files.

Answer 2

How about this?

#!/bin/bash

failed=0

find . -type f -name "*.pl" | while read file; do
   if [ -e "$file" ] && [ -r "$file" ]; then
     sed -i~ "1d" "$file"
     if [ $? != 0 ]; then
        echo "sed returns $? on ($file)"
        (( failed++  ))
     fi
   else
      echo "warning ($file) not exists, or not readable"
      (( failed++ ))
   fi
done

echo "failed execution: $failed"

It is more secure to use sed -i~ here. Sed saves the old file as file~.

Answer 3

GNUly:

find . -type f -name '*.pl' -size +0c -print0 > list &&
  count=$(grep -cz . < list) &&
  stdbuf -oL xargs < list -r0 sed -i -e '1{w /dev/stdout' -e 'd;}' |
    awk -v c="$count" '{printf "processed file %d/%d\r", NR, c}
                       END{print ""}'