12

Apparently, a bracket expression in BSD Awk which contains a character class will ignore any further characters after the character class:

MacOS $ cat file.txt
_
-
.
a
B
8
:
;
@
~
,
MacOS $ awk '/[@~.[:alnum:]:;-]/' file.txt 
.
a
B
8
@
~
MacOS $ awk '/[-;:@~.[:alnum:]]/' file.txt 
-
.
a
B
8
:
;
@
~
MacOS $ awk '/[^@~.[:alnum:]:;-]/' file.txt 
_
-
:
;
,
MacOS $ awk '/[^-;:@~.[:alnum:]]/' file.txt 
_
,
MacOS $ 

On GNU Awk (shown on Ubuntu 16.04), the behavior is different; other characters in the bracket expression are handled the same regardless of whether they come before or after the character class:

Linux $ cat file.txt
_
-
.
a
B
8
:
;
@
~
,
Linux $ awk '/[@~.[:alnum:]:;-]/' file.txt 
-
.
a
B
8
:
;
@
~
Linux $ awk '/[-;:@~.[:alnum:]]/' file.txt 
-
.
a
B
8
:
;
@
~
Linux $ awk '/[^@~.[:alnum:]:;-]/' file.txt 
_
,
Linux $ awk '/[^-;:@~.[:alnum:]]/' file.txt 
_
,
Linux $ 

Is this documented anywhere? Or, if it is a bug, is it a known bug? (And if it is a known bug, is it fixed in later versions of Awk?)


What should I do with this discovery? Is there somewhere I should open a bug report?

11
  • 8
    MacOS is a poor choice of reference since many of the userland utilities haven't been updated for more than a decade. I'd use FreeBSD for a reference (in preference to NetBSD and OpenBSD), but one of those is more likely to have an updated awk, etc. Apr 6, 2017 at 0:03
  • 1
    Indeed: try LANG=C awk ... versus LANG=xx.UTF-8. Apr 6, 2017 at 1:17
  • 2
    It's not really an answer - it's still a bug, it's just one that doesn't show up in the C locale. I don't know about any of the other questions. Apr 6, 2017 at 1:23
  • 2
    FWIW: awk, mawk and gawk are consistent in their behaviour on OpenBSD and does the right things.
    – Kusalananda
    Apr 20, 2017 at 20:22
  • 1
    I think I just stumbled over the same bug: echo 'foo:bar: info' | awk '/^[:[:alpha:]]+: /' outputs foo:bar: info as expected while echo 'foo:bar: info' | awk '/^[[:alpha:]:]+: /' outputs nothing unless I prefix it with LANG=C. awk --version outputs awk version 20070501. GNU awk produces the expected output regardless of the order inside the bracket expression.
    – Ed Morton
    Dec 20, 2017 at 14:58

1 Answer 1

1

Test on FreeBSD 11.2-RELEASE matched the correct results, and matching your results on a recent GNU/Linux.

Perhaps the behavior is a more general case of the bug described here: https://github.com/onetrueawk/awk/issues/45

As far as filing a bug, since it seems that the bug per se is already fixed, and the remaining problem is that Apple is so far behind "upstream", you could try this (though I have not gone through with it before): https://developer.apple.com/bug-reporting/

Note the very last option, which might be a plan B, if we can safely presume that in their current Beta releases, they still haven't updated the UNIX userland from upstream.

2
  • Apple's unwillingness to update userland command line tools could possibly stem from not wanting to break bug-for-bug backward compatibility (it usually does, and there is some merit in that), and this is partly why Homebrew (and similar projects) exist.
    – Kusalananda
    Nov 22, 2019 at 16:46
  • @Kusalananda that is a good point, and something to keep in mind. Such as it is, if many, many customers communicate through multiple channels that they care about it and prefer the bug fixed, maybe, just maybe, it will sort of bubble up in some review queue and eventually outweigh that concern, or at perhaps get tested for absence of actual (vs suspected/presumed) impact.
    – Justin
    Nov 22, 2019 at 17:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.