8

I am in a directory in which I have two text files:

$ touch test1.txt
$ touch test2.txt

When I try to list the files (with Bash) using some pattern it works:

$ ls test?.txt
test1.txt  test2.txt
$ ls test{1,2}.txt
test1.txt  test2.txt

However, when a pattern is produced by a command enclosed in $(), only one of patterns work:

$ ls $(echo 'test?.txt')
test1.txt  test2.txt
$ ls $(echo 'test{1,2}.txt')
ls: cannot access test{1,2}.txt: No such file or directory

What's going on here? Why the pattern {1,2} does not work?

  • 4
    Brace expansion isn't performed within single or double quotes – Sergiy Kolodyazhnyy Jun 6 at 6:47
  • 3
    @SergiyKolodyazhnyy The point of the question is that the ? is also quoted, and gets expanded after $(...) substitutes it, but the brace expansion doesn't. – Michael Homer Jun 6 at 6:48
  • 1
    @muru No, that's not the same issue. Here the order of expansions doesn't matter, what matters is which expansion takes place in which context. I wouldn't be surprised if this question was a duplicate, but I couldn't find it. – Gilles Jun 6 at 7:00
  • 1
    @mosvy Ksh and bash do expansions in the same order, but ksh does brace expansion in a case where bash doesn't do it at all. Zsh-with-globsubst does the same expansions as bash, but in a different order. – Gilles Jun 6 at 20:59
  • 1
    @Gilles no they don't. As documented and easily demonstrated, ksh (and zsh) will perform the brace expansion just before globbing. zsh-with-globsubst won't perform any brace expansion at all on the results of $-expansions: zsh -o globsubst -c 'a=/e*; b={/b*,/v*}; echo $a; echo $b'. – mosvy Jun 6 at 22:14
17

It's a combination of two things. First, brace expansion is not a pattern that matches file names: it's a purely textual substitution — see What is the difference between `a[bc]d` (brackets) and `a{b,c}d` (braces)? . Second, when you use the result of a command substitution outside double quotes (ls $(…)), what happens is only pattern matching (and word splitting: the “split+glob” operator), not a complete re-parsing.

With ls $(echo 'test?.txt'), the command echo 'test?.txt' outputs the string test?.txt (with a final newline). The command substitution results in the string test?.txt (without a final newline, because command substitution strips trailing newlines). This unquoted substitution undergoes word splitting, yielding a list consisting of the single string test?.txt since there are no whitespace characters (more precisely, no characters in $IFS) in it. Each element of this one-element list then undergoes conditional wildcard expansion, and since there is a wildcard character ? in the string the wildcard expansion does happen. Since the pattern test?.txt matches at least one file name, the list element test?.txt is replace by the list of file names that match the patterns, yielding the two-element list containing test1.txt and test2.txt. Finally ls is called with two arguments test1 and test2.

With ls $(echo 'test{1,2}'), the command echo 'test{1,2}' outputs the string test{1,2} (with a final newline). The command substitution results in the string test{1,2}. This unquoted substitution undergoes word splitting, yielding a list consisting of the single string test{1,2}. Each element of this one-element list then undergoes conditional wildcard expansion, which does nothing (the element is left as is) since there is no wildcard character in the string. Thus ls is called with the single argument test{1,2}.

For comparison, here's what happens with ls $(echo test{1,2}). The command echo test{1,2} outputs the string test1 test2 (with a final newline). The command substitution results in the string test1 test2 (without a final newline). This unquoted substitution undergoes word splitting, yielding two strings test1 and test2. Then, since neither of the strings contains a wildcard character, they're left alone, so ls is called with two arguments test1 and test2.

  • 3
    Note that pdksh and ksh93 do perform brace expansion upon expansions (before globbing; not with noglob, but in the case of ksh93, still when braceexpand is turned off!) – Stéphane Chazelas Jun 6 at 7:10
  • You seem to forgot .txt in second explanation. – val Jun 7 at 16:21
10

The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and filename expansion.

Brace expansion won't happen after command substitution. You can use eval to force another round of expansion:

eval echo $(echo '{1,2}lala')

It's result is:

1lala 2lala
6

That problem is very specific to bash, and it's because they decided in bash to separate the brace expansion from filename expansion (globbing), and to perform it first, before all the other expansions.

From the bash manpage:

The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, arithmetic expansion, and command substitution (done in a left-to-right fashion); word splitting; and pathname expansion.

In your example, bash will only see your braces after it had performed the command substitution (the $(echo ...)), when it's just too late.

This is different from all the other shells, which perform the brace expansion just before (and some even as a part of) pathname expansion (globbing). That includes but is not limited to csh where brace-expansions were first invented.

$ csh -c 'ls `echo "test{1,2}.txt"`'
test1.txt test2.txt
$ ksh -c 'ls $(echo "test{1,2}.txt")'
test1.txt  test2.txt

$ var=nope var1=one var2=two bash -c 'echo $var{1,2}'
one two
$ var=nope var1=one var2=two csh -c 'echo $var{1,2}'
nope1 nope2

The latter example is the same in csh, zsh, ksh93, mksh or fish.

Also, notice that brace expansion as part of globbing is also available via the glob(3) library function (at least on Linux and all the BSDs), and in other independent implementations (eg. in perl: perl -le 'print join " ", <test{1,2}.txt>').

Why that was done differently in bash has probably a story behind it, but FWIW I wasn't able to find any logical explanation, and I find all the post-hoc rationalizations unconvincing.

  • 3
    Note that perl used to invoke csh to expand globs, so it's not surprising that it still recognises the same globbing operators as csh – Stéphane Chazelas Jun 6 at 20:29
1

Please try:::

ls $(echo test{1,2}\.txt)

With a BackSlash. It works now. Also remove the like the earlier poster said, the quotes. The Dot is not for matching pattern, but to be taken literally as Period here.

  • (1) The question asks “What’s going on here?  Why [does] the pattern {1,2}” behave the way it does?  The question does not ask “How can I get a command using {1,2} to behave the way the command with ? works?”  You are answering the wrong question.   (2) The backslash has nothing to do with it.  Your command works the way it does because you have removed the quotes that were in the command in the question. – G-Man Jun 8 at 20:34
0

It works if you remove the quotes

$ ls $(echo test{1,2})
test1  test2
  • 9
    The expansion is now happening before the command substitution, which I don't think is what the question is asking for (compare what's happening with ?). – Michael Homer Jun 6 at 6:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.