stream editing tools: output what's scraped

Question

I was just wondering if it was at all possible when using different stream editing and text processing/manipulation tools; such as grep, cut, sed, awk, perl etc. Are there ways to output (in the form of a file or STDOUT) the inverse of the output (To put it another way, what ever was not extracted when running the command) in a universal way i.e. with another application, rather than using the tools features and optional flags to accomplish it.

So for example:

This would print lines in a file that match the regex:

sed '/regexp/!d

This would print lines in a file that do not match the regex:

sed '/regexp/d'

Is it possible to simply run only one of these commands and also have the data that is not contained STDOUT to be also outputted somewhere as well, without using sed's options, but rather a universal method that could then be applied to an awk or grep command.

Could you do something like this, using conn and output the sed command to a file and then compare the original file against the outputted file and output the difference. Something like (also I'm sure this that the syntax is incorrect but I hope the logic is attainable):

comm <(sort originalFile) <(sort sedProcessedFile) > originalFileMinussedProcessedFile

source

But not only apply this to sed but any sort of text/stream manipulation tool, or at least awk and perl.

Answer 1

There's no general method for this. Filters are just reading and writing streams, they're not modifying the stream in place, and the relationship between them is totally arbitrary and dependent on the particular tool. So there's no way a general-purpose mechanism could know that something was "removed" from the input, and therefore save it to be copied to the output.

But tools like awk and perl can write to arbitrary files by themselves, so they can do it with explicit code:

awk '/regexp/ { print } !/regexp/ { print > "non-matching.txt" }' > matching.txt