How to reduce the greediness of a regular expression in AWK?

Question

I want to do non-greedy pattern (regular expression) matching in awk.  Here is an example:

echo "@article{gjn, Author =   {Grzegorz J. Nalepa}, " | awk '{ sub(/@.*,/,""); print }'

Is it possible to write a regular expression that selects the shorter string?

@article{gjn,

instead of this long string?:

@article{gjn, Author =   {Grzegorz J. Nalepa},

I want to get this result:

 Author =   {Grzegorz J. Nalepa},

---

I have another example:

echo ",article{gjn, Author =   {Grzegorz J. Nalepa}, " | awk '{ sub(/,[^,]*,/,""); print }'
      ↑                                                              ↑^^^^^

Note that I changed the @ characters to comma (,) characters in the first position of both the input string and the regular expression (and also changed .* to [^,]*).  Is it possible to write a regular expression that selects the shorter string?

, Author =   {Grzegorz J. Nalepa},

instead of the longer string?:

,article{gjn, Author =   {Grzegorz J. Nalepa},

I want to get this result:

,article{gjn

Answer 1

If you want to select @ and up to the first , after that, you need to specify it as @[^,]*,

That is @ followed by any number (*) of non-commas ([^,]) followed by a comma (,).

That approach works as the equivalent of @.*?,, but not for things like @.*?string, that is where what's after is more than a single character. Negating a character is easy, but negating strings in regexps is a lot more difficult.

A different approach is to pre-process your input to replace or prepend the string with a character that otherwise doesn't occur in your input:

gsub(/string/, "\1&") # pre-process
gsub(/@[^\1]*\1string/, "")
gsub(/\1/, "") # revert the pre-processing

If you can't guarantee that the input won't contain your replacement character (\1 above), one approach is to use an escaping mechanism:

gsub(/\1/, "\1\3") # use \1 as the escape character and escape itself as \1\3
                   # in case it's present in the input
gsub(/\2/, "\1\4") # use \2 as our maker character and escape it
                   # as \1\4 in case it's present in the input
gsub(/string/, "\2&") # mark the "string" occurrences

gsub(/@[^\2]*\2string/, "")

# then roll back the marking and escaping
gsub(/\2/, "")
gsub(/\1\4/, "\2")
gsub(/\1\3/, "\1")

That works for fixed strings but not for arbitrary regexps like for the equivalent of @.*?foo.bar.

Answer 2

There are already several good answers providing work-arounds for awk's inability to do non-greedy matches, so I'm providing some information on an alternative way to do it using Perl Compatible Regular Expressions (PCRE). Note that most simple "match and print" awk scripts can easily be re-implemented in perl using the -n command-line option, and more complex scripts can be converted with the a2p Awk to Perl translator.

Perl has a non-greedy operator which can be used in Perl scripts and anything that uses PCRE. For example, also implemented in GNU grep's -P option.

PCRE is not identical to Perl's regular expressions, but it is very close. It is a popular choice of a regular expression library for many programs, because it's very fast, and the Perl enhancements to extended regular expressions are very useful.

From the perlre(1) man page:

   By default, a quantified subpattern is "greedy", that is, it will match
   as many times as possible (given a particular starting location) while
   still allowing the rest of the pattern to match.  If you want it to
   match the minimum number of times possible, follow the quantifier with
   a "?".  Note that the meanings don't change, just the "greediness":

       *?        Match 0 or more times, not greedily
       +?        Match 1 or more times, not greedily
       ??        Match 0 or 1 time, not greedily
       {n}?      Match exactly n times, not greedily (redundant)
       {n,}?     Match at least n times, not greedily
       {n,m}?    Match at least n but not more than m times, not greedily

Answer 3

This is an old post, but the following information might be useful for others.

There is a way, admittedly crude, to perform non-greedy RE matching in awk. The basic idea is to use the match(string, RE) function, and progressively reduce the size of the string until the match fails, something like (untested):

if (match(string, RE)) {
    rstart = RSTART
    for (i=RLENGTH; i>=1; i--)
        if (!(match(substr(string,1,rstart+i-1), RE))) break;
    # At this point, the non-greedy match will start at rstart
    #  for a length of i+1
}

Answer 4

For general expressions, this can be used as a non-greedy match:

function smatch(s, r) {
    if (match(s, r)) {
        m = RSTART
        do {
            n = RLENGTH
        } while (match(substr(s, m, n - 1), r))
        RSTART = m
        RLENGTH = n
        return RSTART
    } else return 0
}

I'm using this based on @JimMellander's answer. smatch behaves like match, returning:

the position in s where the regular expression r occurs, or 0 if it does not. The variables RSTART and RLENGTH are set to the position and length of the matched string.

Answer 5

There isn't a way in awk to do non-greedy matching. You may be able to get the desired output, though. sch's suggestion will work for that line. If you can't rely on a comma, but "Author" is always the start of the what you want, you could do this:

awk '{ sub(/@.*Author/,"Author"); print }'

If the number of characters preceding Author is always the same, you could do this:

awk '{ sub(/@.{21}/,""); print }'

You just need to know what your data looks like across the whole set.

Answer 6

There is always a way. The given problem can be solved fairly easily by using commas as separator.

echo "@article{gjn2010jucs, Author =   {Grzegorz J. Nalepa}, " |
awk -F, '{sub(/^[ \t]/, "", $2); print $2}'

When the numbers of fields vary something slightly better is usually needed. In such case finding a stop words often pays off, as you can cut anything out from the line by using them. Within context of the example here's what I mean by stop words.

echo "@article{gjn2010jucs, Author =   {Grzegorz J. Nalepa}, " |
awk  '{sub(/.*Author/, "Author", $0); sub(/},.*/, "}", $0); print $0}'

Answer 7

I know this is an old post . But here is something just using awk as OP as requested:
A=@article{gjn2010jucs, Author = {Grzegorz J. Nalepa},
echo $A |awk 'sub(/@[^,]*/,"")'

Output:
, Author = {Grzegorz J. Nalepa},