I have the following text from which I need to remove some wdiff output.

text='Иса Мәсіхтің елшісі Петірден [-(-] осы күнәкар дүниеде [-)-] жат жерлік болып, Понти, Ғалатия, [-Қападоқия, Азия және Бітүния аймақтарында шашыраған [ сенушілерге дұғай сәлем ].-] {+Қападоқия… https://t.co/.......... [Петірдің 1 1:1-5]+}'

I'm trying to remove a [- -] block of text with the non-greedy perl regex \[-.*?-\], but it is matching the previous blocks too:

$ perl -pe 's|\[-.*?-\] {\+(\S+… https://t.co/.*)\+}|\1|' <<<"$text"
Иса Мәсіхтің елшісі Петірден Қападоқия… https://t.co/.......... [Петірдің 1 1:1-5]

Expected output:

$ perl -pe 's|\[-.*?-\] {\+(\S+… https://t.co/.*)\+}|\1|' <<<"$text"
Иса Мәсіхтің елшісі Петірден [-(-] осы күнәкар дүниеде [-)-] жат жерлік болып, Понти, Ғалатия, Қападоқия… https://t.co/.......... [Петірдің 1 1:1-5]

I’ve simplified your input string to qABxBCzABxBCDEFw, where

A represents [
B represents -
C represents ]
D represents {\+
E represents the text between the +s (including the URL)
F represents \+}
Lower case letters represent everything else.

So here are some substitute commands run against the input:

                                  Command                          Output
0. Input text:                                                     qABxBCzABxBCDEFw
1. Non-greedy:                   's|AB.*?BCD(E)F|\1|'              qEw
2. Greedy:                       's|AB.*BCD(E)F|\1|'               qEw
3. Restricted Non-greedy:        's|AB[^B]*?BCD(E)F|\1|'           qABxBCzEw
4. Restricted Greedy:            's|AB[^B]*BCD(E)F|\1|'            qABxBCzEw
5. Constrained Non-greedy:       's|(.*)AB.*?BCD(E)F|\1\2|'        qABxBCzEw
6. Constrained Greedy:           's|(.*)AB.*BCD(E)F|\1\2|'         qABxBCzEw

Command 1 is the one you tried.  Command 2 is the same thing, but not non-greedy.  As you’re aware, they produce the same result.

It seems to me that non-greedy applies only to the length of the text matched by something like .*It doesn’t affect the starting point.  Regex matches always begin as soon as they can.  So, when you say AB.*?BC (i.e., \[-.*?-\]), it’s matching the first [- on the line.  Then, you expect, it’s matching the shortest possible string of any characters ending with -].  You might expect that this would be [-(-].  But look closer: the regex is AB.*?BCD(E)F, so it must match the shortest possible string of any characters ending with -] {+.  This forces it to consume everything up almost to the URL.

I have written four alternative commands that seem to do what you want.  The “Restricted” ones (3 and 4) search, not for AB.*BC, but for AB[^B]*BC; in words, ABBC with no Bs in the “…”.  In your case, that means [--] with no -s in the “…”.  The “Constrained” ones (5 and 6) force the AB.*BC match to start as late as possible by putting a greedy .* in front of them.  It turns out not to matter whether you do the AB.*BC part as greedy or non-greedy in these, either.


You have to use a negative lookaround:

perl -pe 's|\[-((?!-\]).)*-\] {\+(\S+… https://t.co/.*)\+}|\2|' <<<$text

The problem is the first occurence of [- matches. Then the non-greediness of the pattern doesn't take the desired effect, it doesn't matter how non-greedy it is. With a negative lookaround you can match everthing except of the string -], then it matches only the occurence before the {+...+} part.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.