Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

How do I split an entire text consisting of only letters into bigrams.

For example:

odd even

od -> de
dd -> ev

This is what I have so far but it does not produce the desired result.

[some source] | tail -n +30 | sed 's/[^A-Za-z\n]//g' | sed 's/\([A-Za-z]\)\([A-Za-z]\)\([A-Za-z]\)\([A-Za-z]\)\([A-Za-z]\)\{\0,1\}/\1\2 -> \3\4, \2\3 -> \4\5, /g' | sed 's/,/,\n/g'
share|improve this question
2  
I don't understand your desired result format: od produces de? Bigrams of the entire string would be in this order (assuming whitespace characters are not legal members of the bigrams): (o,d) (d,d) (d,e) (e,v) (v,e) (e,n). Considering they don't need to have parantheses and commas for easy parsing, you could have: od dd de ev ve en. Is this going to be done only using sed? –  Herman Torjussen Jan 13 '12 at 14:44

2 Answers 2

up vote 3 down vote accepted

Try next sed script:

Content of infile:

odd even
one test        of              bigrams

Content of script.sed:

## Inside square brackets there are two characters: space and tab.
## The instruction deletes them of the line.
s/[     ]*//g

## Label 'b'.
:b

## Copy line to 'hold space'.
h

## Get first bigram.
s/\(..\)\(..\).*/\1 -> \2/

## If last substitution succeed, continue to label 'a'.
ta

## Here last substitution failed: It means that line has less than four
## characters to extract a bigram, so read next line.
b

## Label 'a'
:a

## Print.
p

## Copy 'hold space' into 'pattern space'.
g

## Delete first character.
s/^.//

## Goto label 'b' to repeat loop.
tb

Run the script:

sed -nf script.sed infile

Result:

od -> de
dd -> ev
de -> ve
ev -> en
on -> et
ne -> te
et -> es
te -> st
es -> to
st -> of
to -> fb
of -> bi
fb -> ig
bi -> gr
ig -> ra
gr -> am
ra -> ms
share|improve this answer

This might work for you:

echo -e "od\ndd\nde\nve" | 
sed '1{x;s/^/oddevenodd/;x};G;/^\(..\)\n.*\1\(..\).*/s//\1 -> \2/'
od -> de
dd -> ev
de -> ve
ve -> no

Is this what you mean?

echo -e "odd even\nthis and that" | 
sed 's/ //g;s/^\(..\)\(.*\)/\1\2\1/;h;:a;s/^\(..\)\(..\).*/\1 -> \2/p;g;/^..../{s/^..//;h;ba};d'
od -> de
de -> ve
ve -> no
th -> is
is -> an
an -> dt
dt -> ha
ha -> tt
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.