Remove lines based on pattern but keeping first n lines that match

Question

I need to remove lines from a text file based on pattern but I need to keep the first n lines that match the pattern.

Input

% 1 
% 2
% 3
% 4
% 5
text1
text2
text3

output

%1
%2
text1
text2
text3

I used sed /^%/d file but it deletes all the lines starting with %, sed 3,/^%/d doesn't work either. I need to keep first n lines of the pattern and delete the rest

Answer 1

If you want to delete all lines starting with % put preserving the first two lines of input, you could do:

sed -e 1,2b -e '/^%/d'

Though the same would be more legible with awk:

awk 'NR <= 2 || !/^%/'

Or, if you're after performance:

{ head -n 2; grep -v '^%'; } < input-file

If you want to preserve the first two lines matching the pattern while they may not be the first ones of the input, awk would certainly be a better option:

awk '!/^%/ || ++n <= 2'

With sed, you could use tricks like:

sed -e '/^%/!b' -e 'x;/xx/{h;d;}' -e 's/^/x/;x'

That is, use the hold space to count the number of occurrences of the patterns matched so far. Not terribly efficient or legible.

Answer 2

I'm afraid sed alone is a bit too simple for this (not that it would be impossible, rather complicated - see e.g. sed sokoban for what can be done).

How about awk?

#!/bin/awk -f
BEGIN { c = 0; }
{
    if (/^%/) {
        if (c++ < 3) {
            print;
        }
    } else {
        print;
    }
}

If you can rely on using recent enough BASH (which supports regular expressions), the awk above can be translated to:

#!/bin/bash -
c=0
while IFS= read -r line; do
    if [[ $line =~ ^% ]]; then
        if ((c++ < 3)); then
            printf '%s\n' "$line"
        fi
    else
        printf '%s\n' "$line"
    fi
done

You can also use sed or grep to do the pattern matching instead of the =~ operator.

Answer 3

A Perl one-liners solution:

# in-place editing
perl -i -pe '$.>2 && s/^%.*//s' filename.txt

# print to the standard output
perl -ne '$.>2 && /^%/ || print' filename.txt

Answer 4

tr '\n' ';' < input | sed 's/% /##/3g' | tr ';' '\n' | sed '/##/d'

I replaced new line characters with ';' to obtain single line string, then turned all but first two occurrences of pattern into ## marking with sed 's/pattern/##/3g' (replace from third to last occurrence of pattern in line), changed back ';' to '\n' and finally removed marked lines.

Answer 5

sed '/^%/{
3,$d}' '% 1 
% 2
% 3
% 4
% 5
text1
text2
text3'

One way of removing the extra lines.

Edit: my answer works under the same condition as Stephane Chazelas's if the % rows doesn't occur first, it won't work.

---

Nerd sniping.

sed -n '/^% [^12]*$/!{
/^% [12][[:digit:]]\{1,\}/n
p}' file.txt

Will work regardless of where the % number string is found in the stream. Any line that starts with % and ends with any number of characters besides 1 or 2, which we negate. That address matches anything besides /% [A-Za-z3-9]*/ leaving an blind spot. Numbers between 10-29 will print still. So we nest a second address to match that range and skip the line.

But awk would still be better.