4

I have problems to understand the string pattern matching with =~ in bash.

I wrote following function (don't be alarmed - it's just experimenting, not a security approach with md5sum):

md5 ()  { 
     [[ "$(md5sum $1)" =~ $2* ]] && echo fine || echo baarr; 
}

and tested it with some input. Here some reference:

md5sum wp.laenderliste
b1eb0d822e8d841249e3d68eeb3068d3  wp.laenderliste

It's unnecessarily hard to compare, if the source for the control sum does not contain the two blanks with the filename already. That's where the observations origins from, but more interesting than the many ways to solve that problem was my observation:

I define a control variable, and test my function with too short, but matching strings:

ok=b1eb0d822e8d841249e3d68eeb3068d3
for i in {29..32}; do md5 wp.laenderliste ${ok:1:$i} ;done 
fine
fine
fine
fine

That's expected and fine, since it is the purpose of the function, to ignore the mismatch of the missing " wp.laenderliste" and therefore even longer mismatches.

Now, if I append random stuff, which does not match, I expect, of course, errors, and get them:

for i in {29..32}; do md5 wp.laenderliste ${ok:1:$i}GU ;done 
baarr
baarr
baarr
baarr

As expected. But when there is only one, last mismatching character, see what happens:

for i in {29..32}; do md5 wp.laenderliste ${ok:1:$i}G ;done 
fine
fine
fine
fine

Is this me, not realizing how this is supposed to work (select is broken), or is there really an off-by-one-error in bash's pattern matching?

Mismatches in the mid of the string matter from count 1:

for i in 5 9 e ; do echo md5 wp.laenderliste ${ok//$i/_} ;done 
md5 wp.laenderliste b1eb0d822e8d841249e3d68eeb3068d3
md5 wp.laenderliste b1eb0d822e8d84124_e3d68eeb3068d3
md5 wp.laenderliste b1_b0d822_8d841249_3d68__b3068d3

for i in 5 9 e ; do md5 wp.laenderliste ${ok//$i/_} ;done 
fine
baarr
baarr

The bash-version:

bash -version
GNU bash, Version 4.3.48(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
Lizenz GPLv3+: GNU GPL Version 3 oder jünger <http://gnu.org/licenses/gpl.html>

Disclaimer: md5sum is only a useful against unintentional mistakes, not against attacks. I don't encourage using it.

And this question is not a search for better solutions or workarounds. It's about the =~ Operator, whether it should act as it does and if so, why.

9

=~ in ([[ ]]) is a regular expression pattern match (or rather, a search, see below). That's different from = (or ==) which uses the same patterns as with filename wildcards.

In particular, the asterisk in regular expressions means "zero or one copies of the preceding unit", so abc* means ab plus zero or more cs.

In your case, the trailing asterisk makes the final character of the function argument optional. In your final example, the pattern becomes ...68d3G*, and since G* matches the empty string, it matches a string like ...68d3. Regexese for "any string" is of .*, or "any character, any number of times".

Note that the regexp match searches for a match anywhere in the string, it doesn't need to be the whole string. So the pattern cde would be found in the string abcdefgh.

You might want to use something like this:

[[ "$(md5sum "$1")" = "$2 "* ]] && echo ok

We don't really need a regular expression match here, and since md5sum outputs the trailing space (plus filename) anyway, we can use that in the pattern to check that we match against the full pattern. So giving the function a truncated hash would not match.

  • Oh, now I feel dirty. I mostly work with sed, when using pattern matching. There of course I have to use .* and know it. Somehow I have learned the now obviously wrong idea, that I have to forget about the dot in the shell - that's only the case for filename matching, right? In case/esac, the dot is needed too? I'm looking so stupid, now! ;) – user unknown Jan 9 '18 at 15:01
  • 1
    @userunknown, case uses the same patterns as with filename matches, so * matches anything, and the dot is just the dot. I think =~ is about the only place in shell that uses regexes. (Then of course ksh/Zsh/Bash have extended globs that have pretty much the same functionality as regexes, but with different syntax. It has to explicitly enabled in Bash, though.) – ilkkachu Jan 9 '18 at 15:05
  • 2
    One could also do [ "$(md5sum < "$1")" = "$2 -" ] to remove the dependency on ksh/zsh/bash (and avoid problems with files whose name starts with -). – Stéphane Chazelas Jan 9 '18 at 15:10
  • Ok, that is a fine excuse. So the rough memory, that pattern matching in the shell is complicated, was at least right. Now I feel much better. :) Now I tried = $2.* and this works too, but "$2 "* would be better. But it was just an experiment, so I won't use anything except the learning experience. – user unknown Jan 9 '18 at 15:13
0

I wouldn't use a regex here, just string comparison:

md5 ()  { 
    sum=$(md5sum "$1" | awk '{print $1}')
    [[ $sum = "$2" ]] && echo fine || echo baarr; 
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.