How to skip file in sed if it contains regex?

Question

I currently use the following simplified command to remove trailing whitespace and add a newline at end of file where needed:

find . -type f -exec sed -i -e 's/[ \t]\+\(\r\?\)$/\1/;$a\' {} \+

As you'll quickly see, this has two problems: It will change binary files and it will add a newline to the end of files with ␍␊ line separators. These modifications are easy to undo or skip when committing in git gui or the like, but I'd like to minimize* the amount of reverting. To that end:

Is there a way to skip the whole file if any line matches a regex in sed?

* I'm aware that there might be binary files without ␀ characters, and there could be files with deliberately mixed newlines or ␀s. But I'm looking for the solution which requires the minimal human intervention. I could conceivably list all the file extensions that I'd like to operate on, but it would be a very long list which would have to be constantly reviewed, and because of name clashes it would still be possible that binary files slip through.

Complicated workaround:

while IFS= read -r -d '' -u 9
do
    if [[ "$(file -bs --mime-type -- "$REPLY")" = text/* ]]
    then
        sed -i -e 's/[ \t]\+\(\r\?\)$/\1/;$a\' -- "$REPLY"
    else
        echo "Skipping $REPLY" >&2
    fi
done 9< <(find . -type f -print0)

Answer 1

If you trust git's point of view on what is a binary file or not, you can use git grep to get a list of non-binary files. Assuming t.cpp is a text file, and ls is a binary, both checked in:

$ ls
t.cpp ls
$ git grep -I --name-only -e ''
t.cpp

The -I option means:

-I
Don't match the pattern in binary files.

To combine that with your sed expression:

$ git grep -I --name-only -z -e '' | \
       xargs -0 sed -i.bk -e 's/[ \t]\+\(\r\?\)$/\1/;$a\'

(-z / xargs -0 to help with strange filenames.)

Check out the git grep man page for other useful options - --no-index or --cached could help depending on exactly what set of files you want to operate on.

Answer 2

Is there a way to skip the whole file if any line matches a regex in sed?

Yes, there is.

# test case for skipping file if a sed regex match succeeds

echo 'Hello, world!' > hello_world.txt
cat hello_world.txt
ls -li hello_world.txt

sed -i -e '/.*Hello.*/{q;}; s/world/WORLD/g' hello_world.txt # skips file
sed -i -e '/.*HeLLo.*/{q;}; s/world/WORLD/g' hello_world.txt

Answer 3

Here's a Perl script that iterates over its arguments (which must be file names) and appends a newline to every file that doesn't end in a newline. Files containing a null byte are skipped. Files that already end in a newline are unmodified. Files that contain a CR get CRLF appended, others get just LF. Untested.

#!/usr/bin/env perl
foreach my $f (@ARGV) {
    open F, "<", $f or die;
    my $last = undef;
    my $cr = 0;
    while (<>) {if (/\0/) {undef $last; break} $last = $_; ++$cr if /\r$/}
    close F;
    if (defined $last && $last !~ /\n\Z/) {
        open F, ">>", $f or die;
        print($cr ? "\r\n" : "\n");
        close F or die;
    }
}