3

I have a text file like so:

<!--START OF FILE -->
random text
<meta> more random text </meta>
x x x x x x x 
more random text
that I dont need 
x x x x x x x

I need everything
from this point
onwards
...

I need to remove everything between <!--START OF FILE --> and the second x x x x x x x like so:

I need everything
from this point
onwards
...

I tried using sed '/<!--START OF FILE -->/,/x x x x x x x/d' test.txt but this removes the block between the first occurence of x x x x x x x which is not what I want.

3

This is quite the opposite of

How to print lines between pattern1 and 2nd match of pattern2?

With sed you'd do something like:

sed -n '/PATTERN1/,$!{         # if not in this range
p;d                            # print and delete
}
/PATTERN2/!d                   # delete if it doesn't match PATTERN2
x;//!d                         # exchange and then, again, delete if no match
: do                           # label "do" (executed only after the 2nd match)
n;p                            # get the next line and print
b do' infile                   # go to label "do"

or, in one line (on gnu setups):

sed -n '/PATTERN1/,$!{p;d;};/PATTERN2/!d;x;//!d;: do;n;p;b do' infile

Sure, it's easier with awk and counters. I'll leave that as an exercise for you...

1

Straightforward awk:

$ awk '/<!--START OF FILE -->/ {a=2}; !a; /x x x x x x x/ && a {a--}' < data

I need everything
from this point
...

It just prints whenever a is zero and decrements it when it sees the x x x ....

Or starting from the actual start of the file instead of a pattern, change the first block to BEGIN {a=2}.

Note that your sample input has an empty line after the second x x x..., and it remains in the output if we stop removing lines at the x x x... line.

0
grep -Pz '(?s)<!--START OF FILE(.*?x x x x x x x){2}\K.*' input.txt

Explanation

  1. grep -Pz

    • -P - Interpret the pattern as a Perl-compatible regular expression (PCRE).
    • -z - process the input.txt as one big line.
  2. (?s)<!--START OF FILE(.*?x x x x x x x){2}\K.*

    • (?s) - Turn on "dot matches newline" for the remainder of the regular expression.
    • .*? - non-greedy matching.
    • {2} - amount of repetitions of the pattern.
    • \K - any previously-matched characters to be omitted from the final matched string.
0

This snippet:

# Utility functions: print-as-echo, print-line-with-visual-space.
pe() { for _i;do printf "%s" "$_i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
pl " Input data file $FILE:"
head -v -n 20 $FILE

pl " Expected output on file $E:"
head -v $E

pl " Results:"
cgrep -V -D -w '<!--START OF FILE -->' +2 +w 'x x x x x x x' 'meta' $FILE

produces:

-----
 Input data file data1:
==> data1 <==
<!--START OF FILE -->
random text
<meta> more random text </meta>
x x x x x x x 
more random text
that I dont need 
x x x x x x x

I need everything
from this point

-----
 Expected output on file expected-output1:

I need everything
from this point
onwards
...

-----
 Results:

I need everything
from this point
onwards
...

This omits (-V) a window beginning (-w) with '...START...', and ending (+w) with the second occurrence (+2) of a string '...x x...' that has the string 'meta' inside the window.

On a system like:

OS, ker|rel, machine: Linux, 3.16.0-4-amd64, x86_64
Distribution        : Debian 8.9 (jessie) 
bash GNU bash 4.3.30

Some details for cgrep:

cgrep   shows context of matching patterns found in files (man)
Path    : ~/executable/cgrep
Version : 8.15
Type    : ELF 64-bit LSB executable, x86-64, version 1 (SYS ...)
Home    : http://sourceforge.net/projects/cgrep/ (doc)

Although one would need to get and compile cgrep, I have had no trouble doing that on 32-bit or 64-bit systems, and it is available on macOS (High Sierra) with brew. The execution time is on a par with GNU grep.

Best wishes ... cheers, drl

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.