Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

as far as i know all unix text processing utilities are reading one line at a time and performing one action on this line.

I have a huge file with a single line of text which contains several tokens im concerned with.

You can think of the content of the file as something like this: xzxzxzzxzxAxzzBxzxCzxxzxxzxzzxzxzAzBzxxxxzzCzxzxzxzxzxxzz

I want to get the two strings between (A and B) and (B and C) for every occurrence of A.*B.*C. In this example my desired output would be this:

xzz xzx

z zxxxxzz

How do i do this?

edit: sorry, i didn't make it clear. A, B and C are long Strings that can only be identified by regular expressions.

share|improve this question
up vote 4 down vote accepted

I'm sure there are many interesting answers using awk,perl,sed, and others. Here is a rather simplistic options that uses tr to turn this problem back into one a problem that we know how to solve--finding a pattern within a line:

 $ tr 'C' '\n' <test.file | sed -n 's/.*A\(.*\)B\(.*$\)/\1 \2/p'

The tr 'C' '\n' command translates any "C" in the input into a newline character. Thus, it is then necessary to just pipe it into a command that will output the text between A and B and between B and the end of the line.

If A, B, and C are regular expressions rather than simple characters, try:

sed -e 's/C/\n/g' < test.file | sed -n 's/.*A\(.*\)B\(.*\)/\1 \2/p'

This uses the same basic idea, but uses sed to create the newlines.

share|improve this answer
sorry, i didn't mention that A, B and C have to be regular expressions. and afaik tr only works with single characters. – Justus1 Dec 12 '10 at 17:44
I updated it with a command that should work even for complex A,B, and C. – Steven D Dec 12 '10 at 17:54

Awk generalizes the notion of lines to record, which can be terminated by any character. Several implementations, such as Gawk, support an arbitrary regular expression as the record separator. Untested:

gawk -vRS='C' 'sub(/.*A/, "") && sub(/B.*/) {print}'
share|improve this answer

If the line can fit into memory, then repeated use of the split function from Perl would work. Otherwise, I would read the file in blocks (with the Perl sysread function) and process each block individually as above -- allowing for stings of interest to cross block boundaries.

share|improve this answer
it easily fits into memory. i don't have any experience with perl. – Justus1 Dec 12 '10 at 17:35

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.