3

I've read up on a few answers here in reference to quoting variables on the shell. Basically, here's what I'm running:

for f in "$(tmsu files)"; do echo "${f}"; done

this has the expected outcome: a list of files. However, I'm pretty sure they're coming out as a single string. After some reading I realized that this is a zsh-ism; it deals with splitting differently than most shells.

So I fired up bash and ran the same command. I was expecting it to split the list, but alas, I got exactly the same result. So now I'm really confused.

  1. Why doesn't bash behave as expected?
  2. How do I get zsh to split lines?

I've tried (zsh): for f in "$(tmsu files)"; do echo "${=f}"; done as well as for f in "$(=tmsu files)"; do echo "${f}"; done neither worked, so obviously I'm mis-understanding how to get zsh to split.

What makes this even worse is at some point I had this doing exactly what I wanted it to, and I can't remember how I did that.

4
  • Does this answer your question? Quoting within $(command substitution) in Bash
    – Haxiel
    Aug 8, 2021 at 4:26
  • 2
    for f in ... iterates over words (where "words" has a rather idiosyncratic definition). If you want to iterate over lines, do not use a for loop; use a while IFS= read -r ... loop instead. Aug 8, 2021 at 7:07
  • @GordonDavisson the link you posted is really helpful, thanks.
    – Harv
    Aug 8, 2021 at 7:38
  • @Haxiel, no, it doesn't, because they want splitting along lines here, while that one focuses on not splitting
    – ilkkachu
    Aug 8, 2021 at 10:01

3 Answers 3

6
for f in "$(tmsu files)"; do echo "${f}"; done

That's more like the wrong thing to do than a zsh-ism. The double-quotes tell the shell to keep the result of the expansion as a single string. In most cases that's what you want, e.g. if you have

filename="foo bar"
ls "$filename"

you want ls to get the filename as a single argument, and not as the two arguments foo and bar. (Those would be different filenames.) Now, the fact that you're using a command substitution there doesn't change that. Though you're right in that quoting doesn't solve your problem in this case, but that construct is awkward to begin with. (In a POSIX shell that is, zsh has tools of its own, of course).

Consider a list of files like:

hello.txt
some silly name with spaces.txt

Using "$(output filenames)" would give you that as one string, not two.
Using $(output filenames) would give you that as six strings, not two.

You could solve that by setting IFS to just the newline, so the latter would split on the newlines and give you the two filenames. But it's a nuisance, has a global effect, and you still need to disable globbing with set -f too in case the file name is funny * name.txt instead.

(This far it's still mostly the same with zsh, since it does split the results of an unquoted command substitution, even if it doesn't split a variable expansion. It doesn't do globbing on the result by default though.)

Now, the reason you can't see the difference is that echo prints all the arguments it gets, joined with single spaces. So echo foo bar and echo "foo bar" produce exactly the same output. Instead, use something like printf "<%s>\n" ..., which lets you see the argument borders more clearly:

$ printf "<%s>\n" foo bar
<foo>
<bar>
$ printf "<%s>\n" "foo bar"
<foo bar>

See also:

4

However, I'm pretty sure they're coming out as a single string.

This is what you get when you use quotes. I can't speak to how zsh is supposed to do it, but the bash manual states as much (section "Word Splitting", under "EXPANSION"):

The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.

(emphasis added). Note that word splitting also does not occur with single quotes; however, substitution and expansion does not occur within them either.

If you want separate arguments out of the command substitution, you need to remove the quotes. However, if tmsu produces any results with spaces, those will also yield undesirable splits (since both spaces and newlines are considered word delimiters), making things a little more complicated. In bash, assuming tmsu gives one result per line, you can do this:

tmsu files |
    while IFS= read -r f
    do
        printf "<%s>\n" "$f"
    done

The read builtin reads a whole line of input. You can give it any number of names. All but the last are assigned based on the word delimiter (space by default), and the last is assigned the remainder of the line, so it's a good way to get around bash's annoying treatment of whitespace in expansion and substitution.

9
  • 1
    Note: zsh and bash and sh all do this the same way.
    – user10489
    Aug 8, 2021 at 4:45
  • 1
    Ugh. Everything I read basically said "use double quotes by default unless you understand what you're doing and have a good reason not to." so this adds to the confusion I have around the topic. I probably just need to learn how to use read. Thanks for taking the time to write such a detailed response.
    – Harv
    Aug 8, 2021 at 7:15
  • Don't feel too bad. Shell languages have really complicated semantics. I can't say I fully understand bash myself, and I have trouble around whitespace and parameter expansion quite often.
    – ddawson
    Aug 8, 2021 at 7:26
  • Right, that helps, thanks. And I think I have the syntax in a form that works although it'll take me a while to figure out how/why it works (and why it breaks in the ways it does).
    – Harv
    Aug 8, 2021 at 7:39
  • "Note that word splitting also does not occur with single quotes..." -- word splitting is completely irrelevant within single quotes, since it only happens for the results of expansions, and those don't happen within single quotes. (Consider something like IFS=x; echoxhi, even though it's not quoted, the word there isn't split since it's not a result of an expansion. (Unless you're in a really ancient pre-POSIX shell, IIRC))
    – ilkkachu
    Aug 8, 2021 at 9:49
4
  1. How do I get zsh to split lines?

With the f parameter expansion flag (details in man zshexpn. It says: Split the result of the expansion at newlines. This is a shorthand for ps:\n:.)

for f in ${(f)"$( tmsu files )"}; do
  print -ru2 -- $f
done

Sending output to stderr to leave stdout for printing pipeable data to (presumably there is more happening than just printing the output).

If your tool can produce output delimited by the ascii null character (aka NUL, \0), then zsh can split that output using the 0 flag (shorthand for ps:\0:) instead of f.

To preserve blank records, add the @ flag and " quotes (e.g. "${(@f)"$( command )"}"). Due to the nature of command substitution, it won't be possible to retrieve trailing newlines.

If you feel like using read in a while loop, something like the following should work in either zsh or bash:

while IFS= read -ru3 -d '' x || [[ $x ]]; do
  {
    printf >&2 '%s\n' "$x"
    z=$x
  } 3<&-
done 3< <(printf 'x\0y\0z')
printf 'z: %s\n' "$z"

No field splitting or backslash interpretation, the loop is not run in a subshell, stdin is still available, and last element is found despite the lack of a trailing delimiter.

1
  • Beware ${(f)"$(cmd)"} removes the empty lines though, so it's not the same thing as looping over the line with while IFS= read -r. Aug 8, 2021 at 16:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .