4

I was playing around with sed after answering another question and I noticed that .+ and .* are not giving the same result when they both match multiple characters in a context address.
The following command1:

sed -E '$!N;/(.+)\n\1/!P;D' <<IN
one
one_more
two
two_more
IN

prints

one_more
two_more

OK, that's the expected output.
Changing the regex from .+ to .* (i.e. from one or more characters to zero or more characters) should give the same result but it does not:

sed -E '$!N;/(.*)\n\1/!P;D' <<IN
one
one_more
two
two_more
IN

prints just one line

two_more

What's going on here ?


1: I'm using ERE for simplicity/readability, the same happens when using BRE

8

That happens because /(.*)\n\1/ is also matching a simple newline (\n: empty string, followed by newline, followed by the same empty string from the beginning).

So it will also match the string one_more\ntwo from your example.

To avoid that, you'll have to anchor your regexps, e.g. sed -E '$!N;/^(.+)\n\1/!P;D' or sed -E '$!N;/^(.*)\n\1/!P;D'.

  • But wouldn't that contradict greediness? – RudiC Oct 24 '18 at 20:32
  • No. It's simply that "one_more\ntwo" will match /(.*)\n\1/, whether greedy or not. To understand what happens in that sed script, add a debugging command after the N eg: sed -E '$!N; {h;s/(.|\n)*/<<&>>/p;x}; /(.*)\n\1/!P; D' ... – mosvy Oct 24 '18 at 20:47
  • @RudiC - greedy means the regex engine attempts to match as much as possible; in this particular case ((.*)\n\1) nothing (the empty string) followed by newline and again by nothing matches and that is sufficient. – don_crissti Oct 24 '18 at 21:10
  • @RudiC one_more\ntwo doesn't match /(.+)\n\1/ at all (since there's no non-empty string that appears on both sides of the NL), but it does match /(.*)\n\1/. Then again, one_more\neno would match /(.+)\n\1/ too, so perhaps the anchors are a good idea. – ilkkachu Oct 24 '18 at 21:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.