Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a simple xml file containing some hyphenated words over page breaks. The input is something like

 ba bla bla hyphe-</page>
 <page>nated bla bla bla

and the output should look like

 bla bla bla</page>
 <page>hyphenated bla bla bla

I am aware of the sed command N, but I don't have control whether my hyphenation occurs on odd or even lines.

Can I do the hyphenation removal as skteched above with sed? Are there alternate ways of doing it (e.g. with other UNIX shell commands or with python or perl)?

EDIT. On request, a real example from my input files:

[...] and vapours, upon the comparison of the air-thermo-</page>
<page>meter with the mercurial thermometer, upon the elastic [...]

EDIT2: Alltho' I picked up the example rather randomly, it is indeed a very nasty one. The wanted output in this case is

 [...] and vapours, upon the comparison of the</page>
<page>air-thermometer with the mercurial thermometer, upon the elastic [...]

i.e. use the space a word separator. The main problem for me is to write a pattern that spans the line break in the original. And yes, the pattern should only remove hyphens immediately preceeding </page>

share|improve this question
    
sorry, my mistake... Let me think how to help you, past comment removed – tachomi Jan 15 at 14:41
    
Can you put some real examples please? – tachomi Jan 15 at 14:43
1  
So, what hyphens do you want removed? What output would you like from that example? Should all hyphens in the file be removed or only some of them? If only some of them, which ones? How can we know? Will it be only the hyphens that are just before a <page>? Please edit your question and clarify. – terdon Jan 15 at 14:57
up vote 3 down vote accepted

Some kind of a monster) With perl it should be easier

cat file
ba bla bla hyphe-</page>
<page>nated bla bla bla
and the output should look like

bla bla bla</page>
<page>hyphenated bla bla bla

It's GNU sed (in some other sed-s -E option is used for extended regular expressions)

sed -nr '/[[:alpha:]]+-<\/[[:alpha:]]+>$/{
N
s!([[:alpha:]]+)-(</[[:alpha:]]+>)\n(<[[:alpha:]]+>)([[:alpha:]]+)!\2\n\3\1\4!}
p' file
ba bla bla </page>
<page>hyphenated bla bla bla
and the output should look like

bla bla bla</page>
<page>hyphenated bla bla bla
share|improve this answer
    
blaming sed for your unfamiliarity w/ it isnt exactly fair. – mikeserv Jan 16 at 18:50

Oneliner in Perl (thanks terdon!):

perl -0 -pe 's/\s+(\S+)-(<\/page>\s+<page>)(\S+)/$2$1$3/g' filename

What it does: matches against regular expression, and uses parts matched to reconstruct your word.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.