How do I split an entire text consisting of only letters into bigrams.

For example:

odd even

od -> de
dd -> ev

This is what I have so far but it does not produce the desired result.

[some source] | tail -n +30 | sed 's/[^A-Za-z\n]//g' | sed 's/\([A-Za-z]\)\([A-Za-z]\)\([A-Za-z]\)\([A-Za-z]\)\([A-Za-z]\)\{\0,1\}/\1\2 -> \3\4, \2\3 -> \4\5, /g' | sed 's/,/,\n/g'
  • 2
    I don't understand your desired result format: od produces de? Bigrams of the entire string would be in this order (assuming whitespace characters are not legal members of the bigrams): (o,d) (d,d) (d,e) (e,v) (v,e) (e,n). Considering they don't need to have parantheses and commas for easy parsing, you could have: od dd de ev ve en. Is this going to be done only using sed? – user13742 Jan 13 '12 at 14:44

Try next sed script:

Content of infile:

odd even
one test        of              bigrams

Content of script.sed:

## Inside square brackets there are two characters: space and tab.
## The instruction deletes them of the line.
s/[     ]*//g

## Label 'b'.

## Copy line to 'hold space'.

## Get first bigram.
s/\(..\)\(..\).*/\1 -> \2/

## If last substitution succeed, continue to label 'a'.

## Here last substitution failed: It means that line has less than four
## characters to extract a bigram, so read next line.

## Label 'a'

## Print.

## Copy 'hold space' into 'pattern space'.

## Delete first character.

## Goto label 'b' to repeat loop.

Run the script:

sed -nf script.sed infile


od -> de
dd -> ev
de -> ve
ev -> en
on -> et
ne -> te
et -> es
te -> st
es -> to
st -> of
to -> fb
of -> bi
fb -> ig
bi -> gr
ig -> ra
gr -> am
ra -> ms
|improve this answer|||||

This might work for you:

echo -e "od\ndd\nde\nve" | 
sed '1{x;s/^/oddevenodd/;x};G;/^\(..\)\n.*\1\(..\).*/s//\1 -> \2/'
od -> de
dd -> ev
de -> ve
ve -> no

Is this what you mean?

echo -e "odd even\nthis and that" | 
sed 's/ //g;s/^\(..\)\(.*\)/\1\2\1/;h;:a;s/^\(..\)\(..\).*/\1 -> \2/p;g;/^..../{s/^..//;h;ba};d'
od -> de
de -> ve
ve -> no
th -> is
is -> an
an -> dt
dt -> ha
ha -> tt
|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.