How can I grep/awk/sed a file looking for some pattern, and print the entire line (including continuation lines if the matched line ends with \?

File foo.txt contains:

thisXXX line \
    has a continuation line
a \
multipleXXX \
continuation \

What should I execute to get (not necessarily in one line, not necessarily removing multiple spaces):

thisXXX line has a continuation line
a multipleXXX continuation line

BTW I'm using bash and fedora21, so it does not need to be POSIX-compliant (but I'll appreciate a solution if it is POSIX)

  • Do you want the search to span over continuation lines? i.e. if you're searching for hello, does hel\␤lo match? – Gilles 'SO- stop being evil' Jun 4 '15 at 22:08
  • @gilles, yes, same as with sh – Carlos Campderrós Jun 5 '15 at 13:43

Another approach using perl to remove newlines that are preceded by \ and whitespace:

$ perl -pe 's/\\\n/ /' file | grep XXX
thisXXX line      has a continuation line
a  multipleXXX  continuation  line

To remove extra spaces, pass it through sed:

$ perl -pe 's/\\\n/ /' file | grep XXX | sed 's/  */ /g'
thisXXX line has a continuation line
a multipleXXX continuation line
  • Since the backslash is supposed to be the last char before the newline, you can remove \s*. And you don't need the g flag – glenn jackman Jun 4 '15 at 18:41
  • @glennjackman not sure I understand. I need the g since I want to remove all extra spaces on the line, the OP's example has a line with multiple continuations. I don't have any \s. – terdon Jun 4 '15 at 22:38
  • I was commenting on your perl command, but Gilles fixed it. – glenn jackman Jun 5 '15 at 10:13
  • @glenn so he did, I was on mobile and completely missed it. Thank you both. – terdon Jun 5 '15 at 10:52

With POSIX sed:

$ sed -e '
' file
  • @don_crissti Pipe this into grep XXX – Gilles 'SO- stop being evil' Jun 4 '15 at 22:10
  • @don_crissti: I don't see matching XXX in requirement. – cuonglm Jun 5 '15 at 1:07
  • @Gilles - no, it doesn't work like that. Change OP's input replacing something with XXX (without a trailing backslash) on first line and then try piping this sed command to grep. You won't get the XXX line in the final output. choroba's solution fails in a similar manner while jimmij's prints the second line too (it shouldn't). – don_crissti Jun 5 '15 at 17:04

Perl to the rescue:

perl -ne 'if (/\\$/) { $l .= $_ }
          else { print $l, $_ if $l =~ /XXX/;
                 $l = "";
          }' foo.txt

$l works as an accumulator. -n processes the input line by line (cf. sed), if the line ends in a backslash, it's added to the accumulator, if not, the accumulator plus the line is printed provided it matches XXX, and the accumulator is emptied.


With pcregrep without changing structure of the lines:

pcregrep -M '^(.|\\\n)*XXX(.|\n)*?[^\\]$' file

My twist:

perl -0777 -ne '                           # read the entire file into $_
    s{ [[:blank:]]* \\ \n [[:blank:]]* }   # join continued lines
     { }gx;
    print grep {/XXX/} split /(?<=\n)/     # print the matching lines
' foo.txt 
thisXXX line has a continuation line
a multipleXXX continuation line

I'd say Perl is the simplest here. It isn't POSIX, though it's in the default installation of most non-embedded unices. If you want POSIX, use awk.

awk '{if (/\\$/) printf "%s" $0; else print}'

This collapses continuation lines. If you want to find patterns that spread over a continuation, pipe this into grep. If you want to match only uninterrupted patterns, let awk accumulate continued lines and do the matching.

awk '{
    if (sub(/\\$/,"")) {
        line = line $0;
    } else {
        if (/XXX/) print;
        line = "";

This is a small improvement to Gilles awk solution (thanks Gilles!), but does require nawk:

nawk '{if (/\\$/) {$0=substr($0,1,length($0)-2); printf "%s",$0} else print}'

This will create a continuous line if the line wraps, but does not include the "\" and space character. (I found this helpful when grepping for PATH statements since the "\" can lead to confusion when interpreting the results.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.