4

I have a line with colon separated values that I want process in awk. Lines are handled differently if variable $4 contains varible $3 at the begin.

So I wrote the expression: $4 ~ /^$3/, but unfortunaltely this does not work, it never matches. What's wrong, how can I use a variable in a regular expression pattern?

This is the full example:

green="$(tput setaf 2)"
red="$(tput setaf 1)"
yellow="$(tput setaf 3)"
normal="$(tput sgr0)"

stacks=$(docker stack ls --format='{{.Name}}')

for stack in ${stacks}; do
    status=$(docker stack ps --filter="desired-state=running" --format="{{.Name}}:{{.Node}}:{{.DesiredState}}:{{.CurrentState}}:{{.Error}}" ${stack})
    if test -z "$status"; then
        echo "${red}$stack$: disabled${normal}"
    else
        awk -F: '                                                                            
            $4 ~ /^$3/ {print "GOOD '"${green}"'" $1 ": " $4 "'"${normal}"'"}                
            $4 !~ /^$3/ {print "BAD '"${yellow}"'" $1 ": " $3 " ≠ " $4 $5 "'"${normal}"'"}   
        ' <<<${status}
    fi
done

Result is always BAD, e.g. here, line:

bind_bind.1:urknall:Running:Running 18 hours ago:

should print GOOD, but prints:

BAD bind_bind.1: Running ≠ Running 18 hours ago
8

/^$3/ is a regular expression that is guaranteed to never match as it matches on records that have 3 after the end of the record (the $ regular expression anchor operator matches at the end of the subject, not to be confused with the $ awk operator that is used to dereference fields by number).

To test whether the third field occurs in the beginning of the fourth field, one could do either a regular expression match with match(), which will return the start position of the match (or -1 if no match was found):

awk -F ':' 'match($4, $3) == 1 { ..."GOOD"... ; next } { ..."BAD"... }'

or, for a string comparison,

awk -F ':' 'substr($4, 1, length($3)) == $3 { ..."GOOD"... ; next } { ..."BAD"... }'
| improve this answer | |
  • The substr solution does exactly, what I tried to achieve, so I marked this as accepted answer, even though other answer also provide good approaches. – Marc Wäckerlin Apr 15 '18 at 10:35
9

You can put the regex on the right side of a ~ in a string, it doesn't have to be a /.../ construct. (The difference might have been related to parsing the RE at runtime or at compile time, but I'm not sure.) And remember that in awk, the dollar sign doesn't mean variable expansion like in shell or Perl, so you need to concatenate $3 to the rest of the string:

The first one matches, the second doesn't:

$ echo 'foo fo+' |awk '$1 ~ "^" $2'
foo fo+
$ echo 'foo o+' |awk '$1 ~ "^" $2'
$

/^$2/ is taken as a regex with a literal $2 in it, and the $ is the end-of-line anchor. Since you can't have anything after the EOL, the RE can never match.

| improve this answer | |
  • I was wondering how /^$2/ would actually be interpreted. Perl would probably interpolate $2 but awk is awfully opaque about what it does with it. – Kusalananda Feb 2 '18 at 11:30
  • 1
    @Kusalananda, for /^$2/, there aren't really ambiguities. There's no way the $ operator ($2 is the $ operator applied to 2, it's not a variable) would be invoked inside /.../, like there's no way /1+2/ would invoke the + addition operator. There can be ambiguities with the / operator. For instance print 1 /2/ 3 is not 1 concatenated with the result of the matching of the 2 regx against $0 concatenated with 3 (for which you'd need 1 (/2/) 3 but 1 divided by 2 divided by 3. And there are ambiguities with backslash. – Stéphane Chazelas Feb 2 '18 at 11:41
  • 4
    @Kusalananda, the $2 ERE is a valid POSIX ERE that is guaranteed to never match. – Stéphane Chazelas Feb 2 '18 at 11:54
  • 2
    @OlivierDulac, that looks like a yet another subtle difference between BRE and ERE. In "9.3.8 BRE Expression Anchoring" it says "A <dollar-sign> shall be an anchor when used as the last character of an entire BRE.", while in "9.4.9 ERE Expression Anchoring" it says "A <dollar-sign> outside a bracket expression shall anchor the expression or subexpression it ends to the end of a string;" (regardless of where the $ appears) – ilkkachu Jun 30 at 12:20
  • 1
    @OlivierDulac, by my reading of "9.3.3 BRE Special Characters" and "9.4.3 ERE Special Characters", \$ should also match a literal dollar sign. Both say "when preceded by a <backslash>, such a [special] character is [a BRE / an ERE] that matches the special character itself". Note that in Perl, you may need to escape the $ even within a bracket expression, since otherwise it can be taken as a variable expansion. – ilkkachu Jun 30 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.