6

I'm looking to find the lines between two matching patterns. If any start or end pattern missing, lines should not print.

Correct input:

a
***** BEGIN *****
BASH is awesome
BASH is awesome
***** END *****
b

Output will be

***** BEGIN *****
BASH is awesome
BASH is awesome
***** END *****

Now suppose END pattern is missing in input

a
***** BEGIN *****
BASH is awesome
BASH is awesome
b

Lines should not print.

I have tried with sed:

sed -n '/BEGIN/,/END/p' input

It prints all data up to the last line if END pattern is missing.

How to solve it?

  • This can be done in Python or Awk, but I doubt it can be done in sed ( or at least not easily) – Sergiy Kolodyazhnyy Jun 6 at 7:32
  • 1
    Can either start and/or end pattern appear multiple times? – Fiximan Jun 6 at 7:34
  • i'm also comfortable with awk. – user356509 Jun 6 at 7:34
  • 1
    yes might be appear multiple times. it may happen start pattern will there, but end pattern is missing, then i don't require that lines. – user356509 Jun 6 at 7:36
  • @SergiyKolodyazhnyy sed is Turing-complete so it can be done in sed, though that doesn't mean it's always a good idea. In this case, though, it's not too bad. – Kevin Jun 6 at 18:01
6
cat input |
sed '/\*\*\*\*\* BEGIN \*\*\*\*\*/,/\*\*\*\*\* END *\*\*\*\*/ p;d' | 
tac |
sed '/\*\*\*\*\* END \*\*\*\*\*/,/\*\*\*\*\* BEGIN *\*\*\*\*/ p;d' |
tac

It works by having tac reverse the lines so that sed can find both delimiters in both orders.

  • Thank you so much. – user356509 Jun 6 at 8:22
9

You can accomplish this as follows:

$ sed -e '
    /BEGIN/,/END/!d
    H;/BEGIN/h;/END/!d;g
' inp

How it works is, for the begin/end range of lines it stores them in hold space. Then deletes till you meet the END line. At which point we recall what is in hold. OTW, we get nothing out. HTH.

5

With pcregrep:

pcregrep -M '(?s)BEGIN.*?END'

That also works if BEGIN and END are on the same line, but not in cases like:

BEGIN 1 END foo BEGIN 2
END

Where pcregrep catches the first BEGIN 1 END, but not the second one.

To handle those, with awk, you could do:

awk '
  !inside {
    if (match($0, /^.*BEGIN/)) {
      inside = 1
      remembered = substr($0, 1, RLENGTH)
      $0 = substr($0, RLENGTH + 1)
    } else next
  }
  {
    if (match($0, /^.*END/)) {
      print remembered $0
      if (substr($0, RLENGTH+1) ~ /BEGIN/)
        remembered = ""
      else
        inside = 0
    } else
      remembered = remembered $0 ORS
  }'

On an input like:

a
BEGIN blah END BEGIN 1
2
END
b
BEGIN foo END
c
BEGIN
bar
END BEGIN
baz END
d
BEGIN
xxx

It gives:

BEGIN blah END BEGIN 1
2
END
BEGIN foo END
BEGIN
bar
END BEGIN
baz END

Both need to store everything from the BEGIN to the following END in memory. So if you have a huge file whose first line contains BEGIN but without an END, the whole file will be stored in memory for nothing.

The only way around that would be to process the file twice, but of course that could only be done when the input is a regular file (not a pipe for instance).

  • Default pcregrep command does not installed. I can't install such command. – user356509 Jun 6 at 7:37
  • @user356509 You can simulate pcregrep with grep -Pzo, see here – Fiximan Jun 6 at 8:12
  • @user356509 pcregrep is usually part of the pcre-tools or pcre2-utils package. – mosvy Jun 6 at 16:33
3

Using sed:

sed '/BEGIN/{b t}; d; :t {N; /END/{p; d;}; b t}'

Explanation:

  • /BEGIN/{b t}; - when /BEGIN/ is matched, switch to the branch labelled t.
  • d; - for other lines, delete and skip the remaining commands
  • :t - the branch labelled t
  • {N; /END/{p; d;}; b t}
    • N - read the next line, append it to current pattern space, then
    • for lines matching /END/, print the accumulated data; skip remaining instructions
    • loop back to branch t.
1

GNU awk approach. The result is achieved via setting particular variables when start header is found. Some variables can be shortened for convenience

$ awk '/BEGIN/{a[i++]=$0;flag=1;next};flag==1{a[i++]=$0;if($0~/END/){print_array=1; nextfile;} }; END{if(print_array) for(j=0;j<=i;j++)print a[j]}' input.txt
***** BEGIN *****
BASH is awesome
BASH is awesome
***** END *****

With missing END flag, the result is null as expected:

$ awk '/BEGIN/{a[i++]=$0;flag=1;next};flag==1{a[i++]=$0;if($0~/END/){print_array=1; nextfile;} }; END{if(print_array) for(j=0;j<=i;j++)print a[j]}' input2.txt
-1

Use the bash && operator ("AND" )

The bash operator && joins two commands. It runs the second command only if the first command produces an exit status of 0. The advantage of using && to conditionally chain commands is that it is a quick and easy-to-remember method for producing shell scripts, compared to the complex syntax that can be required for accomplishing a task in one pass of a single command.

Since the exit status of grep can be 0 (when it finds a matching pattern) or 1 (when it doesn't find a match), you can use grep and && to check a file for a pattern before printing any lines:

file="file.txt" ; grep -q -E '^\*{5} END \*{5}$' "${file}" && sed -n -r '/^\*{5} BEGIN \*{5}$/,/^\*{5} END \*{5}$/p' "${file}"

CAUTION: This example assumes that the BEGIN pattern will occur before the END pattern. This may be a safe assumption, since the question does not show examples where END occurs before BEGIN. If a robust method for dealing with complications is not necessary, the developer's time is better spent on other tasks.

The same code, formatted for easier reading:

file="file.txt"

grep -q -E '^\*{5} END \*{5}$' "${file}" && \
sed -n -r '/^\*{5} BEGIN \*{5}$/,/^\*{5} END \*{5}$/p' "${file}"

In this example, -q tells grep to print no output; this does not affect the exit codes. The -E tells grep that we are using extended regular expressions (in this case, to shorten \*\*\*\*\* to \*{5}).

If grep detects the END pattern, then the sed command runs. The -r does for sed what -E does for grep. Using -n with p tells sed to print from the line containing the BEGIN pattern to the line containing the END pattern, as you tried in the question.

If the asterisks and spaces in the BEGIN and END patterns don't matter, the code is shorter:

file="file.txt" ; grep -q 'END' "${file}" && sed -n '/BEGIN/,/END/p' "${file}"

The same shorter example, formatted for easier reading:

file="file.txt"

grep -q 'END' "${file}" && \
sed -n '/BEGIN/,/END/p' "${file}"
  • Downvoter, please point out errors in this method so that I can improve my answer. – Gaultheria Jun 8 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.