What's the best way to take a segment out of a text file?

Question

What's a good way of extracting say, lines 20 -45 out of a huge text file. Non-interactively of course!

Answer 1

Even simpler:

sed -n '20,45p;45q' < textfile

The -n flag disables the default output. The "20,45" addresses lines 20 to 45, inclusive. The "p" command prints the current line. And the q quits after printing the line.

Answer 2

you could try:

cat textfile | head -n 45 | tail -n 26

or

cat textfile | awk "20 <= NR && NR <= 45" 

update:

As Mahomedalid pointed out, cat is not necessary and a bit redundant, but it does make for a clean, readable command.

If cat does bother you, a better sollution would be:

<textfile awk "20 <= NR && NR <= 45"

Answer 3

This is not an answer but can't post it as a comment.

Another (very fast) way to do it was suggested by mikeserv here:

{ head -n 19 >/dev/null; head -n 26; } <infile

Using the same test file as here and the same procedure, here are some benchmarks (extracting lines 1000020-1000045):

mikeserv:

{ head -n 1000019 >/dev/null; head -n 26; } <iplist

real    0m0.059s

Stefan:

head iplist -n 1000045 | tail -n 26

real    0m0.054s

These are by far the fastest solutions and the differences are negligible (for a single pass) (I tried with different ranges: a couple of lines, millions of lines etc).

Doing it without the pipe might offer a significant advantage, however, to an application which needed to seek over multiple ranges of lines in similar fashion, like:

for  pass in 0 1 2 3 4 5 6 7 8 9
do   printf "pass#$pass:\t"
     head -n99 >&3; head -n1
done <<1000LINES 3>/dev/null
$(seq 1000)
1000LINES

...which prints...

pass#0: 100
pass#1: 200
pass#2: 300
pass#3: 400
pass#4: 500
pass#5: 600
pass#6: 700
pass#7: 800
pass#8: 900
pass#9: 1000

...and only reads the file through the one time.

---

The other sed/awk/perl solutions read the whole file and since this is about huge files, they're not very efficient. I threw in some alternatives that exit or quit after the last line in the specified range:

Stefan:

awk "1000020 <= NR && NR <= 1000045" iplist

real    0m2.448s

vs.

awk "NR >= 1000020;NR==1000045{exit}" iplist

real    0m0.243s

dkagedal (sed):

sed -n 1000020,1000045p iplist

real    0m0.947s

vs.

sed '1,1000019d;1000045q' iplist

real    0m0.143s

Steven D:

perl -ne 'print if 1000020..1000045' iplist

real    0m2.041s

vs.

perl -ne 'print if $. >= 1000020; exit if $. >= 1000045;' iplist

real    0m0.369s

Answer 4

ruby -ne 'print if 20 .. 45' file

Answer 5

Since sed and awk were already taken, here is a perl solution:

perl -nle "print if ($. > 19 && $. < 46)" < textfile

Or, as pointed out in the comments:

perl -ne 'print if 20..45' textfile