Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

A little extended problem from "cat line x to line y on a huge file":

I have a huge file (2-3 GB). I'd like to cat/print only from the line having "foo:" to the line having "goo:". Assume that "foo:" and "goo:" only appear once in a file; "foo:" proceeds "goo:".

So far this is my approach:

  • First, find the line with "foo:" and "goo:": grep -nr "foo:" bigfile
  • Returns 123456: foo: hello world! and 654321: goo: good bye!
  • Once I know these starting and ending line numbers, and the difference (654321-123456=530865) I can do selective cat:
  • tail -n+123456 bigfile | head -n 530865

My question is that how can I effectively substitute the line number constants with expressions (e.g., grep ...)?

I can write a simple Python script but want to achieve it using only combining commands.

share|improve this question

4 Answers 4

sed -n '/foo/,/goo/p;/goo/q' <bigfile

That would print only those lines. If you wanted the line numbers you'd add an =.

sed -n '/foo/=;/goo/=;//q' <bigfile

The q is important because it quits the input when it is called - else sed will continue to read the infile through to the end.

If you don't want to print foo/goo lines you can do instead:

With GNU sed:

sed -n '/foo/,/goo/!d;//!p;/goo/q
' <<\DATA
line1
foo 
line3
line4
line5
goo 
line7
DATA

OUTPUT

line3
line4
line5

And with any other:

sed -n '/foo/G;/\n/,/goo/!d;//q;/\n/!p 
' <<\DATA
line1
foo 
line3
line4
line5
goo 
line7
DATA    

OUTPUT

line3
line4
line5

Either way, though, this also quits its input as soon as it encounters the last line in your search.

share|improve this answer
    
what's the advantage of that longer sed command over sed -n '/foo:/!{ /goo:/,/foo:/!p; }' bigFile? (Not being argumentative; generally curious.) –  HalosGhost Sep 2 at 3:04
    
Ahh, good to know. Thanks! –  HalosGhost Sep 2 at 3:07
    
@HalosGhost - I edited it. Thanks for the inspiration - your own answer gets my vote. –  mikeserv Sep 2 at 3:26
1  
@HalosGhost - thanks again. –  mikeserv Sep 2 at 3:43
1  
sed -n '/foo/,/goo/!d;//!p;/goo/q' won't work with all sed implementations (//!p will only match on goo with some). –  Stéphane Chazelas Sep 2 at 6:24

If you are okay with abandoning your current approach of using something in subshells to get the line numbers and allowing another utility to print the file, this can be accomplished in pure awk with little difficulty:

If you wish to print the lines between foo: and goo: and not the lines themselves, then you can use the following (picked up from here originally):

awk '/goo:/ { exit }; flag; /foo:/ { flag = 1 }' bigFile

The above exits when it sees the end token (goo:), prints if flag is true, and sets flag to true (1, actually) when it reaches the opening token (foo:).

If, however, you would prefer to include the token lines in the output, the command is actually even simpler, as @jasonwryan mentioned:

awk '/foo:/,/goo:/' bigFile

If you are hell-bent on only getting the line numbers and not actually printing the file with the same utility, then you can get the line numbers of the start and end tokens like so:

awk '/foo:|goo:/ { print NR }' bigFile
share|improve this answer
4  
Or, if you want the matches as well: awk '/foo/,/goo/' bigfile –  jasonwryan Sep 2 at 2:50

Alternative sed one:

sed '/foo/,$!d;/goo/q'
share|improve this answer

To substitute the constants with expressions, you can use command substitution.

To substitute the output of a command into an expression, use $(command)

In this case, the appropriate command line is:

tail -n+$(grep -nr "foo:" bigfile | cut -d':' -f1) bigfile | \
head -n$(($(grep -nr "goo:" bigfile | cut -d':' -f1)-$(grep -nr "foo:" bigfile | cut -d':' -f1)+1))

This will print all lines from the line containing foo: to the line containing goo:, inclusive.

share|improve this answer
2  
This reads the input file at least 4 times. You would do better to grep both in one $(( arithmetic )) evaluation statement and save the results into variables. –  mikeserv Sep 2 at 3:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.