0

I have a variable that contains a list of strings, one per line, to be looped over in a for...in...do...done command.

I move regularly between bourne shell and zsh. As far as I can understand it, zsh doesnt by default split words out of a a string at newline or whitespace; bourne shell does. So commands like for list_item in $list; do... fail in zsh whereas they'd work in bourne shell, or with literal text rather than variable. Ive tried playing with IFS= in the command, and quoting the variable, but can't seem to make progress.

Is there a single syntax for a loop over the words/lines in the string, that will work unmodified in both shells, for ease? if not, what's best practice?

4
  • 1
    Do you have some control on that original variable? You say this is a newline-separated list of strings, presumably in one single string. Can it be an array instead? Or anything else deemed more convenient?
    – xhienne
    Oct 14, 2023 at 12:45
  • 2
    Do you really mean bourne shell or do you maybe mean bash instead? The bourne shell is very old and not often used these days. Bash is not the bourne shell, it is the bourne again shell. And even sh on most modern systems is going to be something like dash or bash.
    – terdon
    Oct 14, 2023 at 13:16
  • 1
    I really did mean Bourne shell. I'm using FreeBSD and the /bin/sh is Almquist/ash (basically Debian's dash shell), described as "a clone of SVR3 Bourne Shell".
    – Stilez
    Oct 14, 2023 at 23:29
  • OP answered my question above in my (deleted) answer: « In this case the variable contents are indeed generated by processing command output, along the lines of VAR="$( command /path-to-dir | grep 'stuff' | sed 'expression' )". But in other circumstances that can't be assumed. »
    – xhienne
    Oct 15, 2023 at 11:01

2 Answers 2

2

You can achieve consistent behavior between bash and zsh by explicitly setting zsh's word-splitting behavior to resemble that of bash with the shwordsplit option. For example:

set -o shwordsplit

var='foo bar baz'
for item in $var; do
    echo "$item"
done

This will make zsh perform word splitting based on the Internal Field Separator (IFS), as bash does.

2
  • 1
    Temporarily setting shell options as a workaround isnt really polite. If something goes wrong, and error trapping isnt 100% perfect, it's got potential to mess with anything else run within the session from then on.
    – Stilez
    Oct 14, 2023 at 23:38
  • set -o shwordsplit gives an error in shells other than zsh. Also in shells other than zsh, you'd need to disable globbing with set -o noglob. Oct 15, 2023 at 21:29
1

The Bourne shell is no longer in use and was pre-POSIX and is not POSIX-compliant. There are shells based on the Bourne shell that are POSIX-compliant: the original Korn shell (before the ksh93 rewrite), as that's what the POSIX specification of sh is based on, and bosh which was based on the OpenSolaris sh (itself based on the SysV shell with a few modifications, itself based on the Bourne shell).

FreeBSD's sh is based on the Almquist shell (itself a mostly compatible opensource clone of the SysV shell with a few extensions including some from ksh such as $(...)), but modified to be made POSIX compliant.

In POSIX sh, to loop over lines in a for loop, you can't use split+glob, as the newline being a whitespace character, for IFS-splitting any sequence of one or more newlines act as a separator, so something like:

IFS='
' # split on newline
set -o noglob # disable the glob part
for line in $multiline_string; do
  printf '<%s>\n' "$line"
don

Would skip the empty lines. Also, zsh doesn't do splitting nor globbing upon unquoted parameter expansions, so you'd need a emulate sh there.

You could construct the "$@" array with the lines of $multiline_string with something like:

NL='
'
set --
while [ -n "$multiline_string" ]; do
  line=${multiline_string%%"$NL"*}
  set -- "$@" "$line"
  multiline_string=${multiline_string#"$line"}
  multiline_string=${multiline_string#"$NL"}
done

for line do
  printf '<%s>\n'
done

Or:

printf '%s\n' "$multiline_string" | {
  set --
  while IFS= read -r line; do
    set -- "$@" "$line"
  done
  for line do
    printf '<%s>\n' "$line"
  done
}

Or you could construct the for loop shell code with something like:

eval '
  for line in '"$(
    awk -v q="'" -F '\n' -v ORS=' ' -- '
      BEGIN {
        n = split(ARGV[1], line)
        for (i = 1; i < n; i++) {
          gsub(q, "&\\\\&&", line[i])
          print q line[i] q
        }
      }' "$multiline_string")"'; do
    printf "<%s>\n" "$line"
  done
'

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .