4

I'm trying to understand why grep -w (version 3.1 of the GNU implementation) matches only the first occurrence of a certain pattern in a line.

Here's an example. I would expect that it would match n1, n2 and n3, but it matches only the first one.

$ echo 'n1=1 n2=2 n3=3' | grep -ow "n[0-9]=*"
n1

Or if I tell it to match only n2 or n3, again it matches the first one, and ignores n3.

$ echo 'n1=1 n2=2 n3=3' | grep -ow "n[23]=*" 
n2

What am I missing here? Is there any explanation for this behavior, or is it some sort of bug in grep?

The idea is to match either:

  1. n[0-9] preceded and followed by a non-word character.
  2. A substring that begin with n[0-9] followed by any number of = characters and ends with a non-word character.

So for instance, if the string is n1=1 n2=== n3=3 n4== n5, the expected result should be:

n1
n2===
n3
n4==
n5

Clarification: I know that the goal can be achieved by grep -ow -e 'n[0-9]' -e "n[0-9]=*", but that's beside the point. The goal of the question is to understand how grep works.

Addition tests

If I add n<num>= to different places in the line (without a following word character after the equal sign), it will match those as well, but again it will ignore n3=3.

$ echo 'n1=1 n2= n3=3 n4=' | grep -ow "n[0-9]=*"
n1
n2=
n4=

Last thing that I've found is that if I add -P to interpret the pattern as a Perl-compatible regular expression, it doesn't seem to keep the -w description that says that the substring "must be either at the end of the line or followed by a non-word constituent character", since it matches n1= even though it's followed by the character 1, which is a word constituent character ("letters, digits, and the underscore").

$ echo 'n1=1 n2= n3=3 n4=' | grep -owP "n[0-9]=*"
n1=
n2
n3=
n4

So it seems that grep -wP searches for a word boundary at the end of the substring rather than a non-word constituent character. It seems equivalent to:

$ echo 'n1=1 n2= n3=3 n4=' | grep -o "\bn[0-9]=*\b"
n1=
n2
n3=
n4
6
  • Your grep -owP example output is different from mine (mine is as I would expect)
    – pLumo
    Mar 3 at 13:58
  • I see ... yes .. sorry, I guess you're right....
    – pLumo
    Mar 3 at 14:49
  • 1
    Mmhh. Since = isn't a word-constituent character, I'm not sure how much sense it makes to have a pattern that can accept trailing =s while at the same time looking for whole word matches. I mean, if it's supposed to match a "word", the matching string would probably need to be such that it only contains characters valid in a "word".
    – ilkkachu
    Mar 3 at 15:57
  • @ilkkachu, the -w, --word-regexp section in the man pages of grep doesn't require that the substring must be comprised of word characters. It says nothing about the pattern itself. It only states what it should be preceded and followed with. So you can match a string that contains non-word characters, as long as it's preceded and followed with non-word characters (or beginning/end of the line).
    – aviro
    Mar 3 at 17:25
  • 1
    @aviro, It says it looks for "matches that form whole words". And ok, fine, if the man page says it accepts non-word characters, then it does. But it doesn't make any sense that a "whole word" could contain "non-word characters".
    – ilkkachu
    Mar 3 at 21:49

2 Answers 2

3

Good catch. This seems indeed to be a bug in grep (Tested with GNU grep 3.4 and 3.7):

grep -ow "n[0-9]=*"
grep -Eow "n[0-9]=*"

return only first match (or probably rather matches only the first one),
while ...

grep -Pow "n[0-9]=*"

... returns all matches as expected.

To report bugs for GNU grep, check here.


However, I cannot confirm your observation that -P [...] doesn't seem to keep the -w description, for me (GNU grep 3.4 and 3.7), the command outputs as expected:

$ echo 'n1=1 n2= n3=3 n4=' | grep -owP "n[0-9]=*"
n1
n2=
n3
n4=
8
  • Regarding the statement that "the pattern doesn't make too much sense", see my comment to my question. Also, your sed example is not relevant, since it's not equivalent to the -w flag in grep.
    – aviro
    Mar 3 at 14:23
  • You're right, removed that bit ... Added a workaround though ;-)
    – pLumo
    Mar 3 at 14:51
  • Thanks, the Perl problem was probably fixed in a later version of GNU grep, I have version 3.1.
    – aviro
    Mar 3 at 15:49
  • FWIW, grep -o '\<n[0-9]=*\>' or even grep -wo '\<n[0-9]=*\>' seems to work fine. Mar 3 at 16:27
  • With the busybox or toybox implementations, grep -ow "n[0-9]=*" finds nothing. The ast-open implementation seems fine in that regard. Mar 3 at 16:29
2

So for instance, if the string is n1=1 n2=== n3=3 n4== n5, the expected result should be:

n1
n2===
n3
n4==
n5

Clarification: I know that the goal can be achieved by grep -ow -e 'n[0-9]' -e "n[0-9]=*"

Not so sure about that either:

u$ grep --version |head -1
grep (GNU grep) 2.27
u$ printf '%s\n' 'n1=1 n2=== n3=3 n4== n5' | grep -ow -e 'n[0-9]' -e "n[0-9]=*"
n1
n2===
n3
n4==
n5

vs.

a$ grep --version |head -1
grep (GNU grep) 3.4
a$ printf '%s\n' 'n1=1 n2=== n3=3 n4== n5' | grep -ow -e 'n[0-9]' -e "n[0-9]=*"
n1
n2===
n4==
n5

Note how n3 is missing with the newer grep. That's on Ubuntu, the result s the same with 3.7.

And with Busybox, the answer is different again:

$ printf '%s\n' 'n1=1 n2=== n3=3 n4== n5' | busybox grep -ow -e 'n[0-9]' -e "n[0-9]=*"
n1
n2
n3
n4
n5

The BSD grep on my Mac also prints n1, n2... etc, but each one twice. For whatever reason.

Like Stéphane said in the comments, using -w for that doesn't seem portable.


The idea is to match either:

  • n[0-9] preceded and followed by a non-word character.

  • A substring that begin with n[0-9] followed by any number of = characters and ends with a non-word character.

This sounds to me like you want something like this in Perl (in this order, so that the one with = has priority):

/ n[0-9]=*(?=\W) | \bn[0-9]\b /x

E.g.

$ printf '%s\n' 'n1=1 n2=== n3=3 n4== n5' |
    perl -lne 'print $& while / n[0-9]=*(?=\W) | \bn[0-9]\b /xg'
n1
n2===
n3
n4==
n5

Except that I'm not exactly sure if that's what you want. If it's n2===x instead of n2===, the output is n2==, since the last = is used to fulfill the "ends with a non-word character" clause. (Or rather, "is followed by", since otherwise the match for n1=1 should be n1= too. That would be n1, followed by zero =s, followed by a non-word character =.)

I think you could use the possessive quantifier =*+ to avoid giving back any of the = signs for that, so:

$ printf '%s\n' 'n1=1 n2===X n3=3 n4== n5' |
    perl -lne 'print $& while / n[0-9]=*+(?=\W) | \bn[0-9]\b /xg'
n1
n2
n3
n4==
n5

In any case, I wonder if what you're trying to do would be better achieved by the simple logic of just splitting the string on whitespace, the substrings on the = signs and then looking at the individual values.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.