2

Below I'm issuing the regular expression (a\b) \1 to grep and inserting a couple of test strings through stdin.

$ grep -E '(a\b) \1'
a a
a a
a ab
a ab

The parts in bold mean that there was a match. The second one, a ab, stumps me. The capturing group can be described in words as "the character a followed by a word boundary".

When processing the string a ab, the regex engine matches the character a, see that it is followed by something which isn't a "word character" and thus matches \b. Then it matches a space. So far so good.

But then, it should be checking if \1 matches ab, and as far as I can tell it shouldn't, because following a in ab we have a word character. I don't understand what's going on!


After accepting an answer, I realized I actually still don't understand what's going on. Building up from the examples above:

$ cat tests
a bab
a ba
a ab
$ grep -E '(\ba\b) \1' tests
a ab

This is telling me that the capture group includes everything except word boundaries at the right edge of the string, which I still don't understand.

2
  • Regarding your edit: That's the same thing all over again. You match a ab because that's the only line that contains a a. The two \b only affects the matching of the initial a at the start of the line.
    – Kusalananda
    Sep 26, 2019 at 8:35
  • @Kusalananda This was stupid of me, thanks.
    – regex_hell
    Sep 26, 2019 at 9:02

1 Answer 1

3

The issue is that \1 refers to the matched text, not a regex. In our case, the matched text is the character a. Since \1 is text, not a regex, it does not care what follows the a. Observe:

$ cat file
a a
a ab
$ grep -E '(a\b) \1' file
a a
a ab

If we want \1 to be a word, add to it a word boundary:

$ grep -E '(a\b) \1\b' file
a a

Because \1\b requires a word boundary after \1, the second line no longer matches.

To demonstrate that \1 is not a regex, let's try:

$ echo '.a' | grep -E '(.)\1'
$ 

But:

$ echo '..' | grep -E '(.)\1'
..
$ 

Thus, \1 matches .. While . is normally regex-active and matches any character, \1 will only match a period.

Documentation

From the GNU grep manual:

The back-reference ‘\n’, where n is a single digit, matches the substring previously matched by the nth parenthesized subexpression of the regular expression. For example, ‘(a)\1’ matches ‘aa’. When used with alternation, if the group does not participate in the match then the back-reference makes the whole match fail. For example, ‘a(.)|b\1’ will not match ‘ba’. When multiple regular expressions are given with -e or from a file (‘-f file’), back-references are local to each expression. [emphasis added.]

5
  • Thanks, John. I simultaneously came to a similar conclusion. I rather concluded that the capture group only captures the text itself. So it's not that \1 can't match a regular expression, but rather that the capture group itself only has text (and as a consequence can't match a regular expression). Some tests in Python seem to support this. Do you think what I'm saying is different than what you claim in your answer? Do you think I'm right? Any chance you can find documentation supporting either claim?
    – regex_hell
    Sep 25, 2019 at 23:47
  • Yes, I think you are right that "the capture group itself only has text." I added a link to GNU's documentation which states that \1 matches, in their word, a "substring". I also added an example to demonstrate that \1 does not behave like a regex.
    – John1024
    Sep 25, 2019 at 23:59
  • Hey, John. Sorry, I had to unnaccept the answer because I realized I actually don't understand what's going on. Can you please check the edit?
    – regex_hell
    Sep 26, 2019 at 8:00
  • Apologies, John. All's good, I was being stupid.
    – regex_hell
    Sep 26, 2019 at 9:03
  • @regex_hell Very good. (This is a subtle and complex subject.)
    – John1024
    Sep 26, 2019 at 17:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .