I'm looking for a program to use an external program to filter the lines of a stream - pretty much a version of grep that, for each line, prints or suppresses it based on whether the specified program exits with a zero exit code (like find's -exec option).

I know that I can do this in shell by using a loop and a subshell:

some-program |(while read line; do
    if predicate "$line"; then
        echo "$line"

What I'm wondering is if there's a program floating around that will let me make this simpler:

some-program |filter predicate
# want negation as well
some-program |filter ! predicate

One would think that an enhanced Sed might support this, asking "does it pass on the pattern space?", but GNU Sed does not seem to have such a facility.

Is there such a program somewhere that I haven't found, or do I just need to do it in shell (or perl)?

  • And predicate is a shell command? Are you sure you want to run a new process for every line in the file?
    – alex
    Jun 22, 2011 at 16:32
  • @alex predicate is a shell command, and running a new process for each line is what I'm looking for. Inefficient, but it for one-offs it'd be convenient. I'm actually writing a full Perl script now for the immediate task, but having such a program handy for future reference would still be useful. Jun 22, 2011 at 16:50
  • I'd take a look at shell functions and eval built-in.
    – alex
    Jun 22, 2011 at 18:27
  • You don't actually have to use a subshell; you can do some-program | while ...; do ...; done just fine.
    – Teddy
    Jun 22, 2011 at 19:40

1 Answer 1


The shell is a perfectly suitable tool for this job. Just take care not to mangle spaces and backslashes.

while IFS= read -r line; do
  if predicate "$line"; then printf '%s\n' "$line"; done

You could also use awk. Be sure to quote each line since it'll be passed to a shell (the snippet below puts single quotes around the line, and replaces single quotes in the line by '\''; \047 is '). Because each command invocation goes through a shell, I expect this to be slower than the pure shell method, even if awk is likely to be faster at parsing lines. But I haven't made any benchmarks.

awk '{quoted=$0; gsub(/\047/, "\047\\\\\047\047")}
     !system("predicate \047" $0 "\047")'
  • printf '%s\n' "$line" is the same as echo "$line", isn't it? Jul 4, 2011 at 4:44
  • 2
    @NeilMayhew Almost, but echo "$line" may mangle its argument in a few cases (if $line is -e or -E or -n; and with some shells it may do backslash interpolation) whereas printf '%s\n' "$line" reliably prints $line followed by a newline. Jul 4, 2011 at 6:24
  • if you want to have a ”program“ called filter, simply create a (globally available) shell function with the desired name, see stackoverflow.com/a/34007275/307637
    – myrdd
    Jun 14, 2019 at 9:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.