I need to fix an error and to replace the second tag </time> with </tags> in an XML file with the following structure:

<tags>substancesummit ss</time>
<tags>substancesummit ss</time>

I'm trying to do it using sed and since I have 2 </time> closing tag per item, my idea is to replace </time><geo> with </tags><geo>.

However there is a next line symbol in between, so I'm using \n but it doesn't work:

sed 's/time>\n<geo>/tags>\n<geo>/g' old.xml > new.xml

Sed processes its input line by line, so a newline character will never spontaneously appear in the input. What you could do is put lines ending in </time on hold; then if the next line begins with <geo>, do the substitution in the previous line. (This is possible in sed, using the “hold space”, but I recommend turning to awk or perl when you need the hold space.)

However, given your sample input, you can just change </time> into </tags> when the line begins with <tags>.

sed -e '/^<tags>/ s!</time>$!</tags>!'

While perhaps the solution to your problem can easily be achieved by other means, the answer to your question is a simple one. sed, by default, works a line at a time on 2 buffers - one persistent across line cycles called the hold space and one refreshed at least once per cycle called the pattern space - and the latter is where all edits are performed.

Look ahead can be gained in one of two ways - you can save old lines and fall behind the line cycle in order to make better use of commands to swap and compare the buffers. This involves command primitives such as [hH] old, [gG]et, exchange - which save to, copy from and swap out the hold buffer respectively - and lower case forms overwrite and uppercase forms append to their target buffer.

Or you can work future lines into a constant edit algorithm in which you consistently remove as many input lines as you read per cycle. This latter would be my preference here - especially because sed makes it so very easy and efficient - especially with the N;P;D commands.

Here's a demo using your example data:

sed '$!N;s/ime\(>\n<geo\)/ags\1/;P;D
'  <<\IN
<tags>substancesummit ss</time>
<tags>substancesummit ss</time>

Next, Print, and Delete, like their lower-case counterparts n;p;d get the next line of input, print, and delete into/from pattern space respectively. Unlike their lowercase counterparts (if a little less unlike in N's case), these three work on newline boundaries rather than pattern space as a whole.

  • N will append the next input line to pattern space following a \newline character.
  • P will print only up to the first occuring \newline character in pattern space.
  • D will delete only up to and including the first occurring \newline in pattern space before quitting the script for the current cycle and queueing up the next with whatever remains in pattern space, or, if nothing remains following its delete action, with the next line awaiting on input as per usual.

These three can work together to expand sed's edit window on a file very simply and efficiently - sed slides through a file printing per cycle only the oldest from a series of lines it consistently deletes and replenishes according to a scripter's instructions - which leaves the sedder in charge of the line cycle.

And a next line lookahead is easily expanded upon. If you wanted a 4-line pattern-space window throughout the script you could do:

sed -e '1{N;N' -e '};N;...;P;D'

...or, perhaps more usefully...

sed -e ':next
        N;b next' -e '}

...in which sed only draws in an input line - and continues to do so until it has enough before executing any other commands - if there are fewer than three \newline characters in pattern space and the current line is not the last. This occurs regardless of whatever the edits made by the subsequent commands might do.


To literary answer the question:

I solve this problem (text to edit spans multiple lines) by a little cheat:

cat input.txt | tr '\n' '@' | sed -e 's/txt@iam@interestedin/iaminterested@intxt/g' | tr '@' '\n' > output.txt

The only thing you have to be sure of is that the character you replace the newline with does not yet exist in your input.

  • You must also be careful that the result is not too long to edit - a pattern space consisting of an entire file is one that can either very difficult or even impossible to work with - a text file is officially only a text file if its newline delimiters are no more than 500 bytes apart. – mikeserv Nov 30 '14 at 20:36
  • @mikeserv : sorry, care to differ. It is after all a stream editor link Although there may be platforms that impose a limit. Maybe you can give an example? – JdeHaan Dec 1 '14 at 7:55
  • it is a stream editor for text files. POSIX imposes the 500 char per line limit. But if you want an example just try your thing on a file with millions of lines - any sed will segfault. – mikeserv Dec 1 '14 at 15:44
  • @mikeserv : I did and do, daily. A syslog of over 5GB (didn't care for the number of lines) is no problem at all for 'my thing' and if you had cared to read the link in my previous reply you would have read that GNU-sed, as do others, imposes "no limit" for the maximum line length. An assertion that is not easily made and neither would go unchallenged for long. POSIX is a minimum standard. Anyone can go beyond its limits. – JdeHaan Dec 1 '14 at 17:27
  • It imposes no limit - but memory does. The entire file must fit into memory at once. And probably more than once - each edit represents an altered version of the line that must also be stored in memory at least until the line is written. It is amazing to me that you do this at all w/ a 5gb file - either you have a good deal of memory or a good deal of swap - but it is horribly inefficient in any case. And there is the problem of the replacement delimiter as you mention. POSIX is a minimum standard yes - that's what a standard is. You make no mention of GNU sed in your answer. – mikeserv Dec 1 '14 at 17:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.