Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm looking for a program to use an external program to filter the lines of a stream - pretty much a version of grep that, for each line, prints or suppresses it based on whether the specified program exits with a zero exit code (like find's -exec option).

I know that I can do this in shell by using a loop and a subshell:

some-program |(while read line; do
    if predicate "$line"; then
        echo "$line"
    fi
done)

What I'm wondering is if there's a program floating around that will let me make this simpler:

some-program |filter predicate
# want negation as well
some-program |filter ! predicate

One would think that an enhanced Sed might support this, asking "does it pass on the pattern space?", but GNU Sed does not seem to have such a facility.

Is there such a program somewhere that I haven't found, or do I just need to do it in shell (or perl)?

share|improve this question
    
And predicate is a shell command? Are you sure you want to run a new process for every line in the file? –  alex Jun 22 '11 at 16:32
    
@alex predicate is a shell command, and running a new process for each line is what I'm looking for. Inefficient, but it for one-offs it'd be convenient. I'm actually writing a full Perl script now for the immediate task, but having such a program handy for future reference would still be useful. –  Michael Ekstrand Jun 22 '11 at 16:50
    
I'd take a look at shell functions and eval built-in. –  alex Jun 22 '11 at 18:27
    
You don't actually have to use a subshell; you can do some-program | while ...; do ...; done just fine. –  Teddy Jun 22 '11 at 19:40
add comment

1 Answer 1

The shell is a perfectly suitable tool for this job. Just take care not to mangle spaces and backslashes.

while IFS= read -r line; do
  if predicate "$line"; then printf '%s\n' "$line"; done
done

You could also use awk. Be sure to quote each line since it'll be passed to a shell (the snippet below puts single quotes around the line, and replaces single quotes in the line by '\''; \047 is '). Because each command invocation goes through a shell, I expect this to be slower than the pure shell method, even if awk is likely to be faster at parsing lines. But I haven't made any benchmarks.

awk '{quoted=$0; gsub(/\047/, "\047\\\\\047\047")}
     !system("predicate \047" $0 "\047")'
share|improve this answer
    
printf '%s\n' "$line" is the same as echo "$line", isn't it? –  Neil Mayhew Jul 4 '11 at 4:44
1  
@NeilMayhew Almost, but echo "$line" may mangle its argument in a few cases (if $line is -e or -E or -n; and with some shells it may do backslash interpolation) whereas printf '%s\n' "$line" reliably prints $line followed by a newline. –  Gilles Jul 4 '11 at 6:24
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.