0

I need to parse a file, and I'm looking to print a segment of data between two specific lines. From a "range start" to "range end" but only if the "range end" is present.

If the source file is:

[This is the start] of some data 
this is information
this is more information
This is does not contain the ending required

[This is the start] of some other data
this is info I want
this is info I want
[This is the ending I was looking for]

It should print:

[This is the start] of some other data
this is info I want
this is info I want
[This is the ending I was looking for]

Using grep I've been able to find the data I need and print upwards, but only by a fix number of lines.

Given that the number of lines of data is not constant, is there a way I can use grep or sed, to work up from the end line to find the next occurrence of a given string and capture the specific range I want?

The "range start" of the data segment should be printed along with any data between the "range start" and "range end" point, and the "range end" match is what determines if the whole range of lines should be printed at all. If a range (data segment) does not have the specified end, it should not be printed. If multiple segments have an end point, then all segments containing an end should be printed. No case exists where the input file will have an end without a start, or multiple ends to a single start.

Print lines between (and including) two patterns does not solve my problem, as it starts printing on the first line matched and keeps printing until the the first end segment is found. I need to print only the segments that contain the specified end statement.

  • Im still working on revising it. It is still worded better than my original. Im trying how to best word the revision. – Erudaki Jul 17 '18 at 22:40
  • 1
    What should be done if two "range start" exist? What if two "range end" exist ? – Isaac Jul 17 '18 at 22:55
5

Using sed:

$ sed -n '/This is the start/{h;d;}; H; /This is the ending/{x;p;}' file
[This is the start] of some other data
this is info I want
this is info I want
[This is the ending I was looking for]

Annotated sed script:

/This is the start/{    # We have found a start
    h;                  # Overwrite the hold space with it
    d;                  # Delete from pattern space, start next cycle
};

H;                      # Append all other lines to the hold space

/This is the ending/{   # We have found an ending
    x;                  # Swap pattern space with hold space
    p;                  # Print pattern space
};

What the script does is to save all lines into the "hold space" (a general purpose buffer in sed), but as soon as we find a "start line", we reset that space. When an "end line" is found, the saved data is printed.

This breaks if an "end line" is found before a "start line", and possibly also if two "end lines" are found with no "start line" in-between.


An awk program that goes through the same procedure as the above sed program:

$ awk '/This is the start/  { hold = $0; next }
                            { hold = hold ORS $0 }
       /This is the ending/ { print hold }' file

(identical output as above)

  • Thank you for the annotation edit. That is something I wish more answers had. Makes it easier to reuse for other situations. This answer does work for multiple start to ends, but not in the cases listed above. – Erudaki Jul 17 '18 at 20:48
  • @Isaac Yes, that looks neater and functionally equivalent. Thanks! – Kusalananda Jul 18 '18 at 5:56
0

With multiple START and END patterns, you can do it like this:

sed 'H;/START/h;/END/!d;x;/START/!d' infile

This will unconditionally accumulate lines in the Hold buffer, overwriting it via h each time a START line is encountered (that is, keeping only the data from the most recent START line), deleting the pattern space if it doesn't contain an END line (the cycle restarts here) otherwise exchanging the buffers and again, deleting the pattern space, this time if it doesn't contain START. Whatever is left is autoprinted.

  • This will leak data when both start and end tags were to be collinear. A d after holding start should fix the leak. – Rakesh Sharma Jul 19 '18 at 0:15
  • @RakeshSharma - there is no "leak" as far as I am concerned: that's the intended result. Depends how you read the question which, after countless edits, is still low quality and leaves room to interpretation. – don_crissti Jul 19 '18 at 8:10
0

Use tac to reverse the order of lines

If you use tac to reverse the file — to print the last line first, and so on — then you can extract the region from the end pattern to the start pattern. Then, use tac again to print the lines of output in forward order.

tac file.txt | awk '/^\[This is the ending I was looking for]/,/^\[This is the start]/ { print $0 }' | tac

The same code, formatted to better fit the screen:

tac file.txt | \
awk '/^\[This is the ending I was looking for]/,/^\[This is the start]/ { print $0 }' | \
tac

The { print $0 } isn't necessary in this particular awk command, since it's the default behaviour:

tac file.txt | \
awk '/^\[This is the ending I was looking for]/,/^\[This is the start]/' | \
tac

Unfortunately, if you are using a Mac, tac is not installed by default.

0

You can do it with ex/vi or ed which can do reverse searches e.g.

  • search (forward) for the end pattern
  • enter "normal" mode an place a mark there
  • search backwards from there for the start pattern
  • print from the current line to the mark

ex.

$ ex file << \EOF
/\[This is the ending I was looking for\]
execute "normal! ma\<esc>"
?\[This is the start\]
.,'a p
EOF    
[This is the start] of some other data
this is info I want
this is info I want
[This is the ending I was looking for]

or

$ ed -s file << \EOF
/\[This is the ending I was looking for\]/;#
ka
?\[This is the start\]?;#
.,'a p
EOF
[This is the start] of some other data
this is info I want
this is info I want
[This is the ending I was looking for]

As a one-liner:

printf "/\[This is the ending I was looking for\]/;#\nka\n?\[This is the start\]?;#\n.,'a p\n" | ed -s file
  • The ex --version that prints "VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Sep 30 2017 18:21:38)" in a debian system has no execute command, it prints: "E319: Sorry, the command is not available in this version" – Isaac Jul 17 '18 at 22:22
0

A solution using awk is:

rstart='^[[]This is the start[]]'
rend='[[]This is the ending I was looking for[]]'

awk '$0~rstart{i=1;a=""}
     $0~rstart,$0~rend && i==1 {a = a ((a=="")?"":ORS) $0}
     $0~rend{i=0;print(a)}
    ' rstart="$rstart" rend="$rend" infile

The square brackets are being matched by a [[] and []] to avoid the use of backslash \\[ (which may fail under some conditions).

The main idea is to use a variable i (include) as a boolean value to either include or not each line in the range to be printed. The whole range is acumulated in the variable a. Separated with the ORS (output Record Separator) if the variable a is not null (((a=="")?"":ORS)).

This will print:

[This is the start] of some other data
this is info I want
this is info I want
[This is the ending I was looking for]

If it is required that the start and end markers should not be printed, then use the same code but exchange lines 1 and 3:

awk '$0~rend{i=0;print(a)}
     $0~rstart,$0~rend && i==1 {a = a ((a=="")?"":RS) $0}
     $0~rstart{i=1;a=""}
    ' rstart="$rstart" rend="$rend" infile

Which will print:

this is info I want
this is info I want

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.