GNU grep; cat
:
{ grep -m1 'pattern' &&
cat || ! cat ./infile
} <./infile
POSIX sed; cat
:
{ sed -ne'/PATTERN/q;H;1h;$!d;x;p'; cat; } <infile
GNU sed; cat
:
{ sed -une'/PATTERN/q;H;1h;$!d;x;p'; cat; } <infile
(just add -u
)
sharing is nice
All of the above commands work because the file-descriptor from which they read()
is their parent process's - the shell's. It does the open()
and its children inherit its descriptors. Here they all address it on stdin. One thing about file-descriptors which tends to differ from almost every other kind of inherited environment is that the child processes can affect the parent environment's file-descriptors.
These all require a regular, lseek()
-able ./infile
- (not counting GNU sed
's -u
nbuffered mode, that is). And this is because each of these processes will still do some buffering, but when they have completed whatever their task is they will lseek()
the descriptor right back to the last point they affected it. Otherwise it would be pretty difficult to get things to line up properly (though dd
can be used to effect this).
And because the same descriptor is also passed to the next child process the shell invokes, and the last child has altered its offset, the next command will begin its input immediately where the last left off. And so when we...
seq 10 >nums
{ grep -m1 5; cat; } <nums
grep
quits input on the first match for 5
after printing only the one match it found, and cat
begins copying stdin to stdout just after the newline character which follows 5.
5
6
7
8
9
10
Another thing about grep
is its return value can easily reveal to us if a match is found in input...
{ grep -m1 pattern &&
cat || ! cat ./infile
} <./infile
...where grep
's returns 0 if any match is found &&
cat
takes over from there, ||
else a different cat
copies the whole of ./infile
to output.
some grep
examples
seq 100 >nums
only_after()(
[ -f "$1" ] && {
>/dev/null \
grep -m1 "$2" &&
cat ||! cat "$1"
} <"$1")
only_after nums '[89]\{2\}'
89
90
91
92
93
94
95
96
97
98
99
100
grep
's return reveals to us whether or not it has consumed all of stdin or not. If it returns true the chances are very good that cat
will have some remaining to work with (the only case where it wouldn't be is if grep
found its first match on the last line of input, in which case, by your rules, it shouldn't print anything anyway). But if it consumes the whole stream looking for a match and fails, it will return false, and so the second ||cat
will just print the whole file.
Like this:
seq 5 >nums
only_after nums 8; echo return: "$?"
1
2
3
4
5
return: 1
some sed
examples
seq 200 >nums
{ sed -une'/190/q;H;1h;$!d;x;p'; cat; } <nums
191
192
193
194
195
196
197
198
199
200
...which stacks every input line in sed
's H
old space until either PATTERN
is found and sed
quits input entirely to leave the rest to cat
, or the $
last line is instead found, at which point sed
p
rints all that it has saved. Like this:
seq 10 >nums
{ sed -une'/190/q;H;1h;$!d;x;p'; cat; } <nums
1
2
3
4
5
6
7
8
9
10
Unfortunately, it's prone to implode depending on memory availability and the sed
implementation. Furthermore, a GNU sed
doesn't typically play very nice with others unless you switch it to -u
nbuffered mode, which can have pretty deleterious effects on performance. A POSIX sed
on the other hand is spec'd to play nicely in this way, and so it is definitely an avenue worth exploring.
For a non-lseek()
-able input (such as a pipe) the following could work similarly:
seq 200 | sed -ne'/195/!{H;1h;$!d;x;:p' -ep -e'};n;bp'
196
197
198
199
200
...or...
seq 3 | sed -ne'/195/!{H;1h;$!d;x;:p' -ep -e'};n;bp'
1
2
3
edit-in-place
If you want to replace ./infile
- in other words edit in-place - then you can actually write it over by buffering it to a temp file first:
{ g=$(grep -m1 pattern) &&
cut -c2- <<IN >./infile
$( printf " %s\n" "$g" &&
paste -d\ /dev/null - )
IN
} <./infile
...which will not take any action at all if the pattern cannot be found - and so never reads ./infile
more than the one time - but for a successful match always fully buffers the processed tail of ./infile
out to a temp file before writing that over ./infile
. More specifically - the only part of the infile which gets written out to the shell's here-document is the part after grep
's match. All of the input that grep
consumes in the matching stays consumed and so it is only the buffered tail-end which winds up in the temporary buffer at all.
What's more, most shells will back their here-documents in /tmp
- which, because /tmp
is very often a tmpfs
on a Linux system, means that on said systems the buffered part never winds up on disk at all. In fairness, though, because of the way the kernel will handle file caching and etc, there probably isn't much difference between writing it to a tmpfs and writing it anywhere else provided you have the memory to cache it anyway. /tmp
is maybe just a little more explicit. The shell also unlink()
's the buffer file before ever writing a byte to it - and so it only exists at all for as long as its read and/or write descriptors remain open. There is nothing to clean up.
all wrapped up
I wrote a little program to do this...
allor()(
set -f; unset o z i m; OPTIND=1 IFS="
"
op() while getopts :i:o:m: O &&
case $O$OPTARG in
([$z]*|m*[!0-9]*|[!imo]*) ! : ;;
(o+) o= O=;; (o-) O= ;;
esac||! o=${o+${o:-$i}} m=${m:-1}
do eval "z=$z${O:-o #} $O=\$OPTARG"||exit
done
op "$@";[ -f "${i:?No input specified!}" ] ||i=
exec < "${i:?Input is not a regular file!}" &&
shift $((OPTIND-(${#O}+1))) &&
z=$( ! { { grep -m$m "$@" 2>&3 |
>&4 sed -ne'$=;$s/^/ /p'
} 3>&1| grep . >&2;} 4>&1 ) &&
set ${z:?No match found!} ${o:+'>"$o"'} &&
case $((m==$1))$o in (0"$i") ! : ;;
(0*) <"$i" eval " cat $3 && ! :" ;;
(1*) <<-i eval " cut -c2- $3"
$( printf %s\\n $2;paste /dev/null -)
i
esac
)
...which adds some option parsing and whatnot. Basically you can pass any argument to grep
that you might like to do - all arguments are passed through verbatim except the first of either -i
or -o
or -m
.
You use -i
and its argument to specify the input file. You can use -o-
to write to stdout - which is the default behavior anyway - or -o+
to edit the -i
file in-place, or -o
and any writable pathname. You use -m
to specify a match count - which is to say that you can get all of a file after -m
count matches, or just all of the file if that many matches cannot be found in input. All arguments from the first which isn't a valid -[iom]
switch, or from the second occurring -[io]
are handed directly to grep
.
It tests whether the requested matches were successful and where output should go before attempting to write it. For example, if the matches were not successful and output is directed back over ./infile
it will not do anything at all and leave ./infile
alone. If the matches were successful and the outfile and infile are the same, it will shorten infile. But if the matches are not successful and output is directed anywhere else it will just cat
input to output.
A little demo:
seq 20 >nums
allor -inums -m2 5
15
16
17
18
19
20
...and...
seq 10 >nums
allor -inums -m2 5; echo return: "$?"
1
2
3
4
5
6
7
8
9
10
return: 1
...and...
seq 20000 >nums
allor -m1999 -inums -o+ 5$; cat nums
19985
19986
19987
19988
19989
19990
19991
19992
19993
19994
19995
19996
19997
19998
19999
20000