4

Bash regular expression fails to compare correctly for $'\x01' when End-Of-String char $ is used. All other byte values (seem to) compare correctly.

Using GNU bash 4.1.5(1). Is this a bug, or is there another way to represent bytes in hex notation, other than $'\...'? ...But it doesn't seem to be the notation, because even a literal char to literal char comparison fails.

This 'fail' only happens when the $'\x01' immediately precedes the End-Of-String $.

Here are some examples:

echo 'non \x01 with ^ and $'
[[      3  =~ ^$'\x33'$ ]]; echo $?  # 0 
[[      3  =~ ^$'\063'$ ]]; echo $?  # 0 
[[ $'\x12' =~ ^$'\x12'$ ]]; echo $?  # 0 
[[ $'\002' =~ ^$'\x02'$ ]]; echo $?  # 0 

echo '\x01 with no ^ or $'
[[ $'\x01' =~  $'\x01'  ]]; echo $?  # 0 
[[ $'\x01' =~  $'\001'  ]]; echo $?  # 0 
[[       =~  $'\001'  ]]; echo $?  # 0   nb. Literal char does not render
[[       =~         ]]; echo $?  # 0   nb. Literal char does not render

echo '\x01 with ^ only'
[[ $'\x01' =~ ^$'\x01'  ]]; echo $?  # 0 
[[ $'\x01' =~ ^$'\001'  ]]; echo $?  # 0 
[[       =~ ^$'\001'  ]]; echo $?  # 0   nb. Literal char does not render
[[       =~ ^       ]]; echo $?  # 0   nb. Literal char does not render

echo '\x01 with ^ and $'
[[ $'\x01' =~ ^$'\x01'$ ]]; echo $?  # 1 
[[ $'\x01' =~ ^$'\001'$ ]]; echo $?  # 1 
[[       =~ ^$'\001'$ ]]; echo $?  # 1   nb. Literal char does not render
[[       =~ ^$      ]]; echo $?  # 1   nb. Literal char does not render

echo '\x01 with $ only'
[[ $'\x01' =~  $'\x01'$ ]]; echo $?  # 1 
[[ $'\x01' =~  $'\001'$ ]]; echo $?  # 1 
[[       =~  $'\001'$ ]]; echo $?  # 1   nb. Literal char does not render
[[       =~  $      ]]; echo $?  # 1   nb. Literal char does not render

echo '\x01 with $ only, but not adjacent to \x01'
[[ $'\x01'c =~  $'\x01'c$ ]]; echo $?  # 0 
[[ $'\x01'c =~  $'\001'c$ ]]; echo $?  # 0 
[[      c =~  $'\001'c$ ]]; echo $?  # 0   nb. Literal char does not render
[[      c =~  c$      ]]; echo $?  # 0   nb. Literal char does not render
7
  • Can you run the code through cat -v? It apparently uses the literal character ^A which disappears on StackOverflow. When I replaced the empty places with ^A, I got 0's everywhere (4.3.33(1)).
    – choroba
    Apr 8, 2015 at 16:35
  • It works for me in GNU bash, version 4.3.30(1)
    – cuonglm
    Apr 8, 2015 at 16:35
  • 1
    I agree it is strange and looks like a bug to me. Same behaviour with bash 4.1.17(9), but it works ok with bash 4.3.11(1). Can you update your bash version?
    – gogoud
    Apr 8, 2015 at 16:36
  • 1
    Oh, here is a question which has an answer that links to the discussion, actually, and in which Stéphane Chazelas sums it up quite nicely anyway. Not sure if it is relevant to this or not (maybe not - \001 should never be a multibyte char) - but it is about the general subject anyway.
    – mikeserv
    Apr 8, 2015 at 16:43
  • 2
    Here is a similar bug was reported.
    – cuonglm
    Apr 8, 2015 at 17:10

1 Answer 1

2

Yes, it was a bug in old versions of bash fixed in bash-4.2.14

And here is the commit which makes the problem go away; make of it what you will.

What is CTLESC? It is defined in syntax.h as #define CTLESC '\001', you see. It's some sort of internal escape somehow involved in expansion. It looks like the bug may be that your \x01 datum is being interpreted as if it were an internally-generated CTLESC or something like that.

commit 25db9a70d4c2ba5c43d4167f231bdd8d760d5a06
Author: Chet Ramey <[email protected]>
Date:   Tue Nov 22 20:02:46 2011 -0500

    Bash-4.2 patch 14

diff --git a/patchlevel.h b/patchlevel.h
index 636be1c..04b423b 100644
--- a/patchlevel.h
+++ b/patchlevel.h
@@ -25,6 +25,6 @@
    regexp `^#define[   ]*PATCHLEVEL', since that's what support/mkversion.sh
    looks for to find the patch level (for the sccs version string). */

-#define PATCHLEVEL 13
+#define PATCHLEVEL 14

 #endif /* _PATCHLEVEL_H_ */
diff --git a/pathexp.c b/pathexp.c
index 42f21e4..f239956 100644
--- a/pathexp.c
+++ b/pathexp.c
@@ -196,7 +196,7 @@ quote_string_for_globbing (pathname, qflags)
    {
      if ((qflags & QGLOB_FILENAME) && pathname[i+1] == '/')
        continue;
-     if ((qflags & QGLOB_REGEXP) && ere_char (pathname[i+1]) == 0)
+     if (pathname[i+1] != CTLESC && (qflags & QGLOB_REGEXP) && ere_char (pathname[i+1]) == 0)
        continue;
      temp[j++] = '\\';
      i++;

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .