Bash regular expression comparison fail for hex byte \x01

Question

Bash regular expression fails to compare correctly for $'\x01' when End-Of-String char $ is used. All other byte values (seem to) compare correctly.

Using GNU bash 4.1.5(1). Is this a bug, or is there another way to represent bytes in hex notation, other than $'\...'? ...But it doesn't seem to be the notation, because even a literal char to literal char comparison fails.

This 'fail' only happens when the $'\x01' immediately precedes the End-Of-String $.

Here are some examples:

echo 'non \x01 with ^ and $'
[[      3  =~ ^$'\x33'$ ]]; echo $?  # 0 
[[      3  =~ ^$'\063'$ ]]; echo $?  # 0 
[[ $'\x12' =~ ^$'\x12'$ ]]; echo $?  # 0 
[[ $'\002' =~ ^$'\x02'$ ]]; echo $?  # 0 

echo '\x01 with no ^ or $'
[[ $'\x01' =~  $'\x01'  ]]; echo $?  # 0 
[[ $'\x01' =~  $'\001'  ]]; echo $?  # 0 
[[       =~  $'\001'  ]]; echo $?  # 0   nb. Literal char does not render
[[       =~         ]]; echo $?  # 0   nb. Literal char does not render

echo '\x01 with ^ only'
[[ $'\x01' =~ ^$'\x01'  ]]; echo $?  # 0 
[[ $'\x01' =~ ^$'\001'  ]]; echo $?  # 0 
[[       =~ ^$'\001'  ]]; echo $?  # 0   nb. Literal char does not render
[[       =~ ^       ]]; echo $?  # 0   nb. Literal char does not render

echo '\x01 with ^ and $'
[[ $'\x01' =~ ^$'\x01'$ ]]; echo $?  # 1 
[[ $'\x01' =~ ^$'\001'$ ]]; echo $?  # 1 
[[       =~ ^$'\001'$ ]]; echo $?  # 1   nb. Literal char does not render
[[       =~ ^$      ]]; echo $?  # 1   nb. Literal char does not render

echo '\x01 with $ only'
[[ $'\x01' =~  $'\x01'$ ]]; echo $?  # 1 
[[ $'\x01' =~  $'\001'$ ]]; echo $?  # 1 
[[       =~  $'\001'$ ]]; echo $?  # 1   nb. Literal char does not render
[[       =~  $      ]]; echo $?  # 1   nb. Literal char does not render

echo '\x01 with $ only, but not adjacent to \x01'
[[ $'\x01'c =~  $'\x01'c$ ]]; echo $?  # 0 
[[ $'\x01'c =~  $'\001'c$ ]]; echo $?  # 0 
[[      c =~  $'\001'c$ ]]; echo $?  # 0   nb. Literal char does not render
[[      c =~  c$      ]]; echo $?  # 0   nb. Literal char does not render

Answer 1

Yes, it was a bug in old versions of bash fixed in bash-4.2.14

And here is the commit which makes the problem go away; make of it what you will.

What is CTLESC? It is defined in syntax.h as #define CTLESC '\001', you see. It's some sort of internal escape somehow involved in expansion. It looks like the bug may be that your \x01 datum is being interpreted as if it were an internally-generated CTLESC or something like that.

commit 25db9a70d4c2ba5c43d4167f231bdd8d760d5a06
Author: Chet Ramey <chet.ramey@case.edu>
Date:   Tue Nov 22 20:02:46 2011 -0500

    Bash-4.2 patch 14

diff --git a/patchlevel.h b/patchlevel.h
index 636be1c..04b423b 100644
--- a/patchlevel.h
+++ b/patchlevel.h
@@ -25,6 +25,6 @@
    regexp `^#define[   ]*PATCHLEVEL', since that's what support/mkversion.sh
    looks for to find the patch level (for the sccs version string). */

-#define PATCHLEVEL 13
+#define PATCHLEVEL 14

 #endif /* _PATCHLEVEL_H_ */
diff --git a/pathexp.c b/pathexp.c
index 42f21e4..f239956 100644
--- a/pathexp.c
+++ b/pathexp.c
@@ -196,7 +196,7 @@ quote_string_for_globbing (pathname, qflags)
    {
      if ((qflags & QGLOB_FILENAME) && pathname[i+1] == '/')
        continue;
-     if ((qflags & QGLOB_REGEXP) && ere_char (pathname[i+1]) == 0)
+     if (pathname[i+1] != CTLESC && (qflags & QGLOB_REGEXP) && ere_char (pathname[i+1]) == 0)
        continue;
      temp[j++] = '\\';
      i++;