4

I have several files like file1, file2 ... etc in the same directory and each file may contain several lines matching PATTERN.
I would like to delete the Nth line from each line matching PATTERN e.g. with N = 3 and file1 content like

1 no match
2 PATTERN
3 same PATTERN
4 no match here
5 no match here either
6 another PATTERN
7 again, no match
8 no
9 last line

the expected output is

1 no match
2 PATTERN
3 same PATTERN
4 no match here
7 again, no match
8 no

Editing the files in-place is a bonus, not a requirement (though there's at least one gnu tool that I know of that could edit them all in one go...)


A similar question was asked here however that is a particular case (there's just a single line matching pattern in each file and the solutions there would only work with multiple lines matching pattern if they're separated by at least N+1 non-matching lines).

  • If a line matching the pattern is to be deleted because of a previous match, should it still delete the following n-th, or not? – guido May 2 '16 at 18:53
  • 1
    @guido - yes, it should always delete the Nth; I think the example is pretty clear. – don_crissti May 2 '16 at 19:11
  • @don_crissti, I agree your example is very clear. It made me realize what you were really asking when my first attempt failed. I think the easiest way around this issue is a double use of tac, as I suggest in my answer. Congrats on a question that's much trickier than it first appears. :) – Wildcard May 3 '16 at 0:00
6

You could use awk for this I believe like so:

awk -vN=3 '/PATTERN/ {skips[FNR+N]=1;} {if(!(FNR in skips)) print;}' <file>

so each time we hit PATTERN we'll record the line that is N away from here, and only print those lines we have not marked for skipping.

with gawk you could use -i inplace as well to do it in place

As you noted, that wouldn't handle multiple files. Of course, you could wrap with a for loop to iterate over all the files, but if there aren't enough to make the commandline too long you could also do it like so:

 awk -vN=3 '{if(FNR==1) split("", skips, ":");} /PATTERN/ {skips[FNR+N]=1;} {if(!(FNR in skips)) print;}' *

where we reset skips to an empty array each time FNR hits 1, so the start of each file.
With gnu awk you could write it as:

gawk -i inplace 'FNR==1{delete nr};/PATTERN/{nr[FNR+3]++};!(FNR in nr)' file*
2

I like a 2-pass mechanism so we can use sed -i:

for file in file1 ...
do sed -i "$file" -e "$(awk <"$file" -v N=3 '/PATTERN/{ print (NR+N) "d" }')"
done
2
for f in file1 file2 file...; do
  sed -i -f <(grep -n PATTERN "$f" | while IFS=: read line rest; do printf "%dd; " $((line+3)); done) "$f"
done

To split that apart:

  1. Loop over file1 file2 file...

  2. build up a sed expression inside the process substitution, to eventually run against the file.

  3. grep outputs line numbers matching PATTERN in the file (along with the actual matching line).

Sample output:

2:2 PATTERN
3:3 same PATTERN
6:6 another PATTERN
  1. the while loop strips the line number off, discarding the matching line, then sending it to printf, incremented by 3

  2. printf prints the targeted line number, followed by the sed d delete command and a separating semicolon.

Sample output (as input for sed):

5d; 6d; 9d;

This method allows for a good amount of flexibility; you could set N=3 and use $((line+N)) as the printf argument.

To account for the in-place edit, I assume a sed that supports -i "in-place" editing.

2

This use case just begs for using ex.

Unfortunately, since deleting the third line after a given line may delete a line containing PATTERN and thus cause the deletion associated with that line to be skipped (or worse, to delete the incorrect line), you need to reverse the file using e.g. tac first. Then you can delete the third line before each instance of PATTERN, and reverse the file once more:

for f in *.txt; do printf %s\\n '%!tac' 'g/PATTERN/-3d' '%!tac' x | ex "$f"; done

If you have tac available I think this is the cleanest solution.


For a fully POSIX compliant solution, making use of my answer to:

You can do it like so:

for f in *.txt; do printf %s\\n '%!sed -n '\''1h;1\!{x;H;};${g;p;}'\' 'g/PATTERN/-3d' '%!sed -n '\''1h;1\!{x;H;};${g;p;}'\' x | ex "$f"; done

Not very readable, but functional.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.