Why does the substitution of newlines using g/re/p only apply to every other line in Vim?

Question

Consider:

foo
bar
....
bar
baz

Then:

:g/bar/s/\n/\r\r/g

unexpectedly returns:

foo
bar

bar
bar

bar
bar

...
baz

when I want newlines after every bar. Yet:

:g/bar/p

returns:

bar
bar
...
bar

as expected and if there are newlines between all the bars I get what I expect--an extra newline after every line containing bar. I solved the problem by via:

:%s/\(bar.*\)\n/\1\r\r/g

but I'd really like to have a better understanding on what's going on.

Answer 1

To understand what's going on at this level of detail, I think you have to reach for the source code. So, citing vim-7.1.314/src/ex_cmds.c (comment at the head of the definition of ex_global()):

This is implemented in two passes: first we scan the file for the pattern and set a mark for each line that (not) matches. secondly we execute the command for each line that has a mark. This is required because after deleting lines we do not know where to search for the next match.

Thus, if you delete a line in the :g'ed command, the mark for this line goes away too. If you create a line in the :g'ed command, it won't be marked, so the command won't execute on it. In your example, line 3 is joined with line 2, so the mark on line 3 disappears, and the next mark after line 2 is the one originally put on line 4.

Also, ex_global sets the global_busy variable, which causes a few commands, including :s, to behave slightly differently. I don't think the difference is relevant here though.

Answer 2

Gilles has answered your “why?”, but I thought you might be interested in a simpler way to do what you wanted.

I want newlines after every bar.

Instead of replacing the line terminating character (thus joining the lines and losing the internal mark, per Gilles’ answer), add a blank line right before the end of each matching line. Use $ to get a zero-width match right before the end of the line and insert a line break there (also no need for the /g modifier since the pattern will only match once per line).

:g/bar/s/$/\r/