72

How can I skip the first 6 lines/rows in a text file (input.txt) and process the rest with awk? The format of my awk script (program.awk) is:

BEGIN {
} 

{ 
process here
} 

END {

}

My text file is like this:

0
3
5
0.1 4.3
2.0 1.5
1.5 3.0
0.3 3.3
1.5 2.1
.
.
.

I want to process the file starting from:

0.3 3.3
1.5 2.1
.
.
.

5 Answers 5

101

Use either of the two patterns:

NR>6 { this_code_is_active }

or this:

NR<=6 { next }
{ this_code_is_active }

Use FNR instead of NR if you have many files as arguments to awk and want to skip 6 lines in every file.

0
51

Try:

awk 'FNR > 6 { #process here }' file
1
  • 15
    Good! But you didn't explain why this is better - for multiple files FNR is the row number in each file, while NR is the number in the whole input (not an issue when piping). Aug 8, 2017 at 5:46
9

You may also skip an arbitrary number of lines at the beginning or the end of the file using head or tail programs.

For your concrete question,

tail input.txt -n+7 | program.awk

will do, provided your program.awk file is executable. Otherwise, you may use

tail input.txt -n+7 | awk -f program.awk

This way, you will spare a comparison for each line and you don't need to change the logic of your AWK code.

tail will start streaming text starting at the seventh line, skipping the six first lines.

This will not be a huge deal in performance, especially if text process is simple thanks to caching. However, for long files and repeated use in cloud environment may save some cost.

2
  • 3
    Correct, but as a rule of thumb, one should avoid piping when you can easily do it with one tool alone. Think of a huge text file piping through both commands just to remove a few lines.
    – Philippos
    Sep 1, 2017 at 11:21
  • 2
    I disagree about avoiding piping. Do one thing and do it well. Adage aside, I do whatever comes to me from memory first. Scripting vs CLI too though. Small sets on a CLI, I go for least effort. Then a script may work on large datasets, or may desire to reduce the scope of future maintenance, so maybe less pipes.
    – Kevin
    Jul 27, 2020 at 20:56
1

Another one:

$ cat print_from.awk

#print 20 lines from line 6 (from your file: 0.3 3.3)
BEGIN{n=0}
/^0.3 3.3.*/ {n=NR; m=n+20}
{
  if (n > 1 && NR < m) {print $1} 
}

execute with:

awk -f print_from.awk your-text-file
1
  • This is a decent solution — to a different problem. The question says (twice!) that the requirement is to skip the first 6 lines. For example, if the first seven lines are ant, bat, cat, dog, elk, fox and giraffe, it should start processing with giraffe. But that’s just an example; you have written an answer that starts processing with giraffe, regardless of the line number. Also, the question says “process the rest”, not “process the next 20.” Dec 30, 2019 at 7:39
-1
for pkg in $(pip list | awk 'FNR > 2 { print $1 }'); do pip install --upgrade $pkg; done
1
  • 1
    Welcome to Unix & Linux! Your answer is a bit terse and could be improved by explaining the issue, i.e., the underlying cause of the problem and why your solution correctly would resolve it for the user in the question.
    – Kusalananda
    Feb 21 at 18:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .