18

I am trying to print the lines using the repetition symbol {n} but it is not working. For. e.g. I want to print all lines whose length is 4 char long

 awk '/^.{4}$/' test_data

The above code is not printing that .How to fix it so that I can use the repetition symbol? I know the alternative like awk '/^....$/' test_data and awk 'length ==3 ' test_data

  • 3
    What distribution are you using? Which awk? – terdon Mar 29 '17 at 12:07
  • 1
    $ awk --version GNU Awk 3.1.7 $ cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.7 (Santiago) – Forever Learner Mar 29 '17 at 12:08
  • 2
    I'd say awk '/^.{4}+$/{print}' <<<$'foods\nbaarsz\nfooo' to match exactly 4 chars. Also as you mentioned yourself, awk 'length($0) == 4' test_data is compatible with almost all awk versions. – Valentin Bajrami Mar 29 '17 at 12:09
  • 4
    Do awk --re-interval '/^.{4}$/' test_data or awk --posix '/^.{4}$/' test_data work? – steeldriver Mar 29 '17 at 12:15
  • Thank you steeldriver. This solved my issue. Upvoted. Thanks again :) – Forever Learner Mar 29 '17 at 12:17
20

According to The GNU Awk User's Guide: Feature History, support for regular expression range operators was added in version 3.0 but initially required explicit command line option

New command-line options:

  • New command-line options:
    • The --lint-old option to warn about constructs that are not available in the original Version 7 Unix version of awk (see V7/SVR3.1).
    • The -m option from BWK awk. (Brian was still at Bell Laboratories at the time.) This was later removed from both his awk and from gawk.
    • The --re-interval option to provide interval expressions in regexps (see Regexp Operators).
    • The --traditional option was added as a better name for --compat (see Options).

In gawk 4.0,

Interval expressions became part of default regular expressions

Since you are using gawk 3.x, you will need to use

awk --re-interval '/^.{4}$/'

or

awk --posix '/^.{4}$/'

or (thanks @StéphaneChazelas) if you want a solution that is portable, use

POSIXLY_CORRECT=anything awk '/^.{4}$/'

(since --posix or --re-interval would cause an error in other awk implementations).

| improve this answer | |
  • Thanks steeldriver, for your time and help. Upvoted and accepted as an answer – Forever Learner Mar 29 '17 at 12:31
  • 4
    It's better to use POSIXLY_CORRECT=anything awk '/^.{4}/' as that makes portable code (a --posix or --re-interval would cause an error in other awk implementations). – Stéphane Chazelas Mar 29 '17 at 12:36
  • Hi Stéphane Chazelas, when I issued the command , $ POSIXLY_CORRECT=anything awk '/^.{4}/' test_data, it printed all the lines. Then I realized the there is no last dollar after repetitions. Thanks for your inputs. Upvoting your comment and solution. Sorry I misunderstood it in the first place due to omission of the $ after repetition. – Forever Learner Mar 29 '17 at 12:39
20

EREs (extended regular expressions as used by awk or egrep) initially didn't have {x,y}. It was first introduced in BREs (as used by grep or sed), but with the \{x,y\} syntax that didn't break backward portability.

But when it was added to EREs with that {x,y} syntax, it did break backward portability as a foo{2} RE was matching something different before.

So some implementations chose not to do it. You'll find that /bin/awk, /bin/nawk and /bin/egrep on Solaris still don't honour it (you need to use /usr/xpg4/bin/awk or /usr/xpg4/bin/grep -E). Same for awk and nawk on FreeBSD (based on the awk maintained by Brian Kernighan (the k in awk)).

For GNU awk, until relatively recently (version 4.0), you had to call it with POSIXLY_CORRECT=anything awk '/^.{4}$/' for it to honour it. mawk still doesn't honour it.

Note that that operator is only syntactic sugar. .{3,5} can always be written ....?.? for instance (though of course {3,5} is a lot more legible, and the equivalent of (foo.{5,9}bar){123,456} would be a lot worse).

| improve this answer | |
  • Thanks again Stéphane Chazelas. Sorry, my bad, I was unable to comprehend your answer initially. Thanks a lot and upvoted. – Forever Learner Mar 29 '17 at 12:42
6

This works as expected with GNU awk (gawk):

$ printf 'abcd\nabc\nabcde\n' | gawk '/^.{4}$/'
abcd

But fails with mawk which is closer to POSIX awk and, AFAIK, is the default on Ubuntu systems:

$ printf 'abcd\nabc\nabcde\n' | mawk '/^.{4}$/'
$ ## prints nothing

So, a simple solution would be to use gawk instead of awk. The {n} notation isn't part of the POSIX BRE (basic regular expression) syntax. That's why grep also fails here:

$ printf 'abcd\nabc\nabcde\n' | grep '^.{4}$'
$

However, it is part of ERE (extended regular expressions):

$ printf 'abcd\nabc\nabcde\n' | grep -E '^.{4}$'
abcd

I don't know which regex flavor is used by mawk or POSIX awk, but I would guess it's BRE. They use an older version of ERE according to Stéphane's answer. In any case, either you are apparently using a version of awk that doesn't implement ERE or your input doesn't actually have any lines with exactly 4 characters. This could happen because of whitespace that you don't see or unicode glyphs, for example.

| improve this answer | |
  • Hi terdon, I want to print the lines which are 4 characters long. Not the first four characters of a line. For example $ grep -E '^.{4}$' test_data , will work but same is not working with awk – Forever Learner Mar 29 '17 at 12:15
  • @CppLearner yes, that's what I'm doing here. What do you mean? – terdon Mar 29 '17 at 12:19
  • @CppLearner, @terdon's solution does only print lines which are 4 characters long. But if you're really only interested in line length, you should just use length($0) which is more efficient than regexes. – Stephen Kitt Mar 29 '17 at 12:19
  • Hi terdon , steeldriver 's solution is what I was looking for. Thanks for your time. Hi Stephen Kitt , As I mentioned in the problem, I already used the length as alternative, I was more interested in knowing why the repetition regex {n} is not working from steeldriver's comment I came to know that I need to use the option of --re-interval or --posix. Thanks for your time. – Forever Learner Mar 29 '17 at 12:23
  • 1
    mawk is not really closer to POSIX awk, and doesn't use BREs. It does use EREs but without the {x,y} operator. – Stéphane Chazelas Mar 29 '17 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.