as far as i know all unix text processing utilities are reading one line at a time and performing one action on this line.

I have a huge file with a single line of text which contains several tokens im concerned with.

You can think of the content of the file as something like this: xzxzxzzxzxAxzzBxzxCzxxzxxzxzzxzxzAzBzxxxxzzCzxzxzxzxzxxzz

I want to get the two strings between (A and B) and (B and C) for every occurrence of A.*B.*C. In this example my desired output would be this:

xzz xzx

z zxxxxzz

How do i do this?

edit: sorry, i didn't make it clear. A, B and C are long Strings that can only be identified by regular expressions.

3 Answers 3


I'm sure there are many interesting answers using awk,perl,sed, and others. Here is a rather simplistic options that uses tr to turn this problem back into one a problem that we know how to solve--finding a pattern within a line:

 $ tr 'C' '\n' <test.file | sed -n 's/.*A\(.*\)B\(.*$\)/\1 \2/p'

The tr 'C' '\n' command translates any "C" in the input into a newline character. Thus, it is then necessary to just pipe it into a command that will output the text between A and B and between B and the end of the line.

If A, B, and C are regular expressions rather than simple characters, try:

sed -e 's/C/\n/g' < test.file | sed -n 's/.*A\(.*\)B\(.*\)/\1 \2/p'

This uses the same basic idea, but uses sed to create the newlines.

  • sorry, i didn't mention that A, B and C have to be regular expressions. and afaik tr only works with single characters.
    – Justus1
    Dec 12, 2010 at 17:44
  • I updated it with a command that should work even for complex A,B, and C.
    – Steven D
    Dec 12, 2010 at 17:54

Awk generalizes the notion of lines to record, which can be terminated by any character. Several implementations, such as Gawk, support an arbitrary regular expression as the record separator. Untested:

gawk -vRS='C' 'sub(/.*A/, "") && sub(/B.*/) {print}'

If the line can fit into memory, then repeated use of the split function from Perl would work. Otherwise, I would read the file in blocks (with the Perl sysread function) and process each block individually as above -- allowing for stings of interest to cross block boundaries.

  • it easily fits into memory. i don't have any experience with perl.
    – Justus1
    Dec 12, 2010 at 17:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .