Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have some Perl files which I have to modify through a simple sed call. Basically, I have to remove the first line of all of them. Currently, I managed to do this :

find <PATH> -type f -name "*.pl" -exec sed -i '1d' {} \;

However, I am using this line in a script and I would it to be a bit more... talkative. Thus, I decided to echo a counter (in real time), that would show how many files have been processed so far.

I know the count of Perl files can be retrieved through

PERL_FILE_COUNT=$(find <PATH> -name "*.pl" | wc -l)

Currently, I have this

remove_first_line()
{
    count=0
    echo -ne "Removing first line of Perl files ..."
    find <PATH> -type f -name "*.pl" -exec sed -i '1d' {} \; >/dev/null 2>&1

    if [ $? -eq 0 ]
        then echo "OK"
        then echo "FAILED"
    fi
}

Now, what I would like as an output, would be something like this:

"Removing first line of Perl files ... 1/209209"

And the value should be automatically updated. But I don't see how to increment the count variable with my find/exec statement. Basically, each time sed finished working a file, it should increment the count variable.

share|improve this question
    
Regarding the find <PATH> -name "*.pl" | wc -l, the wc command output will only be executed once the pipe is closed, not for every line. –  Ouki Mar 27 at 15:04

3 Answers 3

up vote 2 down vote accepted

If you have bash 4, consider using globstar instead. It gives you recursive globbing.

shopt -s globstar
perlz=( **/*.pl ) # */ Hack to fix syntax highlighting
totes="${#perlz[@]}"
i=0
for file in "${perlz[@]}"; do
    printf 'Removing first line of Perl files … %d/%d\r' $((++i)) $totes
    ed -s "$file" <<< $'1d\nw' # You can use `sed` if you want to, but ed is an actual file editor
done
echo # print a final newline

This solution will work across files with crazy characters in their names and avoids a subshell.

But if bash 4 isn't an option, you can recreate this solution using find -exec +:

find . -name '*.pl' -exec bash -c 'totes=$#
  i=0
  for file; do
    printf "Removing first line of Perl files … %d/%d\r" $((++i)) $totes
    ed -s "$file" <<< $'\''1d\nw'\'' # Avoid these leaning toothpicks by putting this
                                     # script in a file.
  done
  echo # print a final newline
' bash {} +

However, this is subject to your system's ARG_MAX (unlike the above), so if the number of files is very large you could still end up with multiple runs over subsets of the files.

share|improve this answer

GNUly:

find . -type f -name '*.pl' -size +0c -print0 > list &&
  count=$(grep -cz . < list) &&
  stdbuf -oL xargs < list -r0 sed -i -e '1{w /dev/stdout' -e 'd;}' |
    awk -v c="$count" '{printf "processed file %d/%d\r", NR, c}
                       END{print ""}'
share|improve this answer

How about this?

#!/bin/bash

failed=0

find . -type f -name "*.pl" | while read file; do
   if [ -e "$file" ] && [ -r "$file" ]; then
     sed -i~ "1d" "$file"
     if [ $? != 0 ]; then
        echo "sed returns $? on ($file)"
        (( failed++  ))
     fi
   else
      echo "warning ($file) not exists, or not readable"
      (( failed++ ))
   fi
done

echo "failed execution: $failed"

It is more secure to use sed -i~ here. Sed saves the old file as file~.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.