4

Bash regular expression fails to compare correctly for $'\x01' when End-Of-String char $ is used. All other byte values (seem to) compare correctly.

Using GNU bash 4.1.5(1). Is this a bug, or is there another way to represent bytes in hex notation, other than $'\...'? ...But it doesn't seem to be the notation, because even a literal char to literal char comparison fails.

This 'fail' only happens when the $'\x01' immediately precedes the End-Of-String $.

Here are some examples:

echo 'non \x01 with ^ and $'
[[      3  =~ ^$'\x33'$ ]]; echo $?  # 0 
[[      3  =~ ^$'\063'$ ]]; echo $?  # 0 
[[ $'\x12' =~ ^$'\x12'$ ]]; echo $?  # 0 
[[ $'\002' =~ ^$'\x02'$ ]]; echo $?  # 0 

echo '\x01 with no ^ or $'
[[ $'\x01' =~  $'\x01'  ]]; echo $?  # 0 
[[ $'\x01' =~  $'\001'  ]]; echo $?  # 0 
[[       =~  $'\001'  ]]; echo $?  # 0   nb. Literal char does not render
[[       =~         ]]; echo $?  # 0   nb. Literal char does not render

echo '\x01 with ^ only'
[[ $'\x01' =~ ^$'\x01'  ]]; echo $?  # 0 
[[ $'\x01' =~ ^$'\001'  ]]; echo $?  # 0 
[[       =~ ^$'\001'  ]]; echo $?  # 0   nb. Literal char does not render
[[       =~ ^       ]]; echo $?  # 0   nb. Literal char does not render

echo '\x01 with ^ and $'
[[ $'\x01' =~ ^$'\x01'$ ]]; echo $?  # 1 
[[ $'\x01' =~ ^$'\001'$ ]]; echo $?  # 1 
[[       =~ ^$'\001'$ ]]; echo $?  # 1   nb. Literal char does not render
[[       =~ ^$      ]]; echo $?  # 1   nb. Literal char does not render

echo '\x01 with $ only'
[[ $'\x01' =~  $'\x01'$ ]]; echo $?  # 1 
[[ $'\x01' =~  $'\001'$ ]]; echo $?  # 1 
[[       =~  $'\001'$ ]]; echo $?  # 1   nb. Literal char does not render
[[       =~  $      ]]; echo $?  # 1   nb. Literal char does not render

echo '\x01 with $ only, but not adjacent to \x01'
[[ $'\x01'c =~  $'\x01'c$ ]]; echo $?  # 0 
[[ $'\x01'c =~  $'\001'c$ ]]; echo $?  # 0 
[[      c =~  $'\001'c$ ]]; echo $?  # 0   nb. Literal char does not render
[[      c =~  c$      ]]; echo $?  # 0   nb. Literal char does not render
  • Can you run the code through cat -v? It apparently uses the literal character ^A which disappears on StackOverflow. When I replaced the empty places with ^A, I got 0's everywhere (4.3.33(1)). – choroba Apr 8 '15 at 16:35
  • It works for me in GNU bash, version 4.3.30(1) – cuonglm Apr 8 '15 at 16:35
  • 1
    I agree it is strange and looks like a bug to me. Same behaviour with bash 4.1.17(9), but it works ok with bash 4.3.11(1). Can you update your bash version? – gogoud Apr 8 '15 at 16:36
  • 1
    Oh, here is a question which has an answer that links to the discussion, actually, and in which Stéphane Chazelas sums it up quite nicely anyway. Not sure if it is relevant to this or not (maybe not - \001 should never be a multibyte char) - but it is about the general subject anyway. – mikeserv Apr 8 '15 at 16:43
  • 2
    Here is a similar bug was reported. – cuonglm Apr 8 '15 at 17:10
1

Yes, it was a bug in old versions of bash fixed in bash-4.2.14

And here is the commit which makes the problem go away; make of it what you will.

What is CTLESC? It is defined in syntax.h as #define CTLESC '\001', you see. It's some sort of internal escape somehow involved in expansion. It looks like the bug may be that your \x01 datum is being interpreted as if it were an internally-generated CTLESC or something like that.

commit 25db9a70d4c2ba5c43d4167f231bdd8d760d5a06
Author: Chet Ramey <chet.ramey@case.edu>
Date:   Tue Nov 22 20:02:46 2011 -0500

    Bash-4.2 patch 14

diff --git a/patchlevel.h b/patchlevel.h
index 636be1c..04b423b 100644
--- a/patchlevel.h
+++ b/patchlevel.h
@@ -25,6 +25,6 @@
    regexp `^#define[   ]*PATCHLEVEL', since that's what support/mkversion.sh
    looks for to find the patch level (for the sccs version string). */

-#define PATCHLEVEL 13
+#define PATCHLEVEL 14

 #endif /* _PATCHLEVEL_H_ */
diff --git a/pathexp.c b/pathexp.c
index 42f21e4..f239956 100644
--- a/pathexp.c
+++ b/pathexp.c
@@ -196,7 +196,7 @@ quote_string_for_globbing (pathname, qflags)
    {
      if ((qflags & QGLOB_FILENAME) && pathname[i+1] == '/')
        continue;
-     if ((qflags & QGLOB_REGEXP) && ere_char (pathname[i+1]) == 0)
+     if (pathname[i+1] != CTLESC && (qflags & QGLOB_REGEXP) && ere_char (pathname[i+1]) == 0)
        continue;
      temp[j++] = '\\';
      i++;

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.