5

I have a simple xml file containing some hyphenated words over page breaks. The input is something like

 ba bla bla hyphe-</page>
 <page>nated bla bla bla

and the output should look like

 bla bla bla</page>
 <page>hyphenated bla bla bla

I am aware of the sed command N, but I don't have control whether my hyphenation occurs on odd or even lines.

Can I do the hyphenation removal as skteched above with sed? Are there alternate ways of doing it (e.g. with other UNIX shell commands or with python or perl)?

EDIT. On request, a real example from my input files:

[...] and vapours, upon the comparison of the air-thermo-</page>
<page>meter with the mercurial thermometer, upon the elastic [...]

EDIT2: Alltho' I picked up the example rather randomly, it is indeed a very nasty one. The wanted output in this case is

 [...] and vapours, upon the comparison of the</page>
<page>air-thermometer with the mercurial thermometer, upon the elastic [...]

i.e. use the space a word separator. The main problem for me is to write a pattern that spans the line break in the original. And yes, the pattern should only remove hyphens immediately preceeding </page>

  • sorry, my mistake... Let me think how to help you, past comment removed – tachomi Jan 15 '16 at 14:41
  • Can you put some real examples please? – tachomi Jan 15 '16 at 14:43
  • 1
    So, what hyphens do you want removed? What output would you like from that example? Should all hyphens in the file be removed or only some of them? If only some of them, which ones? How can we know? Will it be only the hyphens that are just before a <page>? Please edit your question and clarify. – terdon Jan 15 '16 at 14:57
3

Some kind of a monster) With perl it should be easier

cat file
ba bla bla hyphe-</page>
<page>nated bla bla bla
and the output should look like

bla bla bla</page>
<page>hyphenated bla bla bla

It's GNU sed (in some other sed-s -E option is used for extended regular expressions)

sed -nr '/[[:alpha:]]+-<\/[[:alpha:]]+>$/{
N
s!([[:alpha:]]+)-(</[[:alpha:]]+>)\n(<[[:alpha:]]+>)([[:alpha:]]+)!\2\n\3\1\4!}
p' file
ba bla bla </page>
<page>hyphenated bla bla bla
and the output should look like

bla bla bla</page>
<page>hyphenated bla bla bla
  • blaming sed for your unfamiliarity w/ it isnt exactly fair. – mikeserv Jan 16 '16 at 18:50
4

Oneliner in Perl (thanks terdon!):

perl -0 -pe 's/\s+(\S+)-(<\/page>\s+<page>)(\S+)/$2$1$3/g' filename

What it does: matches against regular expression, and uses parts matched to reconstruct your word.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.