5

I'm trying to print only the <N>th line before a search pattern. grep -B<N> prints all the <N> lines before the search pattern. I saw the awk code here that can print only the <N>th line after the search pattern.

awk 'c&&!--c;/pattern/{c=N}' file

How to modify this to print only the <N>th line before each line that matches pattern ? For example, here is my input file

...
...
   0.50007496  0.42473932  0.01527831
   0.99997456  0.97033575  0.44364198
Direct configuration=     1
   0.16929051  0.16544726  0.16608723
   0.16984300  0.16855274  0.50171112
...
...
   0.50089841  0.42608090  0.01499159
   0.99982054  0.97154975  0.44403547
Direct configuration=     2
   0.16931296  0.16553376  0.16600890
   0.16999941  0.16847055  0.50170694  
...

I need a command that can give me back the 2nd line before the search string Direct configuration. I'm trying to run this in SUSE-Linux

  • 1
    Any reason you can't just grep ... | head -1 ? – thrig May 16 '16 at 16:30
  • I'm not sure I understand. There are tens of thousands of occurence of the search pattern in the file and I need only the Nth line before every occurence – WanderingMind May 16 '16 at 16:33
  • 1
    @thrig that would only return the first line, if you have more than one match they'll get pruned in the head execution. – Bratchley May 16 '16 at 16:36
  • 1
    How about grep -B<N> ... | grep -A1 '^--$' | grep -v '^--$', to cut it down to only the first line of the resulting context? – JigglyNaga May 16 '16 at 16:44
  • 3
    The 'N lines after' version works by starting a counter when the match is seen, and printing when the count reaches 0. To do 'N lines before', you'd have to save all of the last N lines somewhere, and print out the first of them when the match is seen. It might be simpler to 'tac' the input file first. – JigglyNaga May 16 '16 at 16:54
11

A buffer of lines needs to be used.

Give a try to this:

awk -v N=4 -v pattern="example.*pattern" '{i=(1+(i%N));if (buffer[i]&& $0 ~ pattern) print buffer[i]; buffer[i]=$0;}' file

Set N value to the Nth line before the pattern to print.

Set patternvalue to the regex to search.

buffer is an array of N elements. It is used to store the lines. Each time the pattern is found, the Nth line before the pattern is printed.

  • 1
    @WanderingMind Then you should accept this answer. Especially since it uses awk as per your OP – Paul Evans May 17 '16 at 13:06
8

That code doesn't work for previous lines. To get lines before the matched pattern, you need to somehow save the lines already processed. Since awk only has associative arrays, I can't think of an equally simple way of doing what you want in awk, so here's a perl solution:

perl -ne 'push @lines,$_; print $lines[0] if /PAT/; shift(@lines) if $.>LIM;' file 

Change PAT to the pattern you want to match and LIM to the number of lines. For example, to print the 5th line before each occurrence of foo, you would run:

perl -ne 'push @lines,$_; print $lines[0] if /foo/; shift(@lines) if $.>5;' file 

Explanation

  • perl -ne : read the input file line by line and apply the script given by -e to each line.
  • push @lines,$_ : add the current line ($_) to the array @lines.
  • print $lines[0] if /PAT/ : print the first element in the array @lines ($lines[0]) if the current line matches the desired pattern.
  • shift(@lines) if $.>LIM; : $. is the current line number. If that is greater than the limit, remove the 1st value from the array @lines. The result is that @lines will always have the last LIM lines.
  • @WanderingMind yes, sorry, my bad. Perl usually assumes $_ when nothing is given but apparently that's not the case for push. I was using it when testing and then removed it for the answer to make it shorter. I should have tested the shorter version. – terdon May 16 '16 at 18:05
  • 2
    @terdon awk does propose classic arrays. Take a look at my answer. – Jay jargot May 16 '16 at 18:06
  • 1
    @Jayjargot yes, I've already upvoted your answer, that's a neat trick. Those are still associative arrays though, it's just that you're using a number as an index. You still can't push or pop them. – terdon May 16 '16 at 18:08
4

Alternative way with sed.

For N=1:

sed '$!N; /.*\n.*pattern/P; D' FILE

For N=2

sed '1N; $!N; /.*\n.*\n.*pattern/P; D' FILE

For the N=2 case, the 1st line this will read the next N-1 lines in the pattern space then start a N;P;D cycle - read another line in and if the last line in the pattern space matches, print the first line in the pattern space then delete it, starting a new cycle.

The downside is that it needs to be modified for different values of N:

For N=3:

sed '1{N;N}; $!N; /.*\n.*\n.*\n.*pattern/P; D' FILE

For N=4:

sed '1{N;N;N}; $!N; /.*\n.*\n.*\n.*\n.*pattern/P; D' FILE

so it quickly becomes cumbersome though for greater values of N you could prepare a script file and pass it to sed.

  • @AlexHarvey - what's wrong with infile as an input file name? – don_crissti Apr 22 at 20:08
  • Nothing, sorry. I see upper case used a lot in modern documentation (Python, Ruby's arg parser for instance). Not always. To me, a nice way to distinguish a placeholder string from a literal string. Shall I change it back? – Alex Harvey Apr 22 at 21:28
3
tac file | awk 'c&&!--c;/pattern/{c=N}' | tac

But this has the same omission as the 'forwards' use case when there are multiple matches within N lines of each other.

And it won't work so well when the input is piped from a running process, but it's the simplest way when the input file is complete and not growing.

  • 2
    What should happen if there are 2 occurrences of pattern less than N lines apart? I think this would squash the previously found one before the associated line is printed. – Eric Renouf May 16 '16 at 17:12
  • @EricRenouf True, the original 'forwards' example misses those, so mine will too. Edited to mention that. – JigglyNaga May 16 '16 at 18:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.