0

If I ls files without a for loop in a remote host everything is fine however, if I capture the ls in a variable and try to echo each file it fails as the variables don't expand/have a value.

What I mean:

IFS=$'\n'
servers=(
  blue
  green
  red
)
for i in ${servers[@]}; do 
   ssh $i "
     ls /dir/xyz_${i}*/details.txt
   "
done
/dir/xyx_blue0/details.txt 
/dir/xyz_blue1/details.txt
/dir/xyx_green2/details.txt 
/dir/xyz_green4/details.txt
/dir/xyx_red1/details.txt 
/dir/xyz_red8/details.txt

But I actually need to loop through the output of ls so I can do things to the files, however the variable doesn't expand:

for i in ${servers[@]}; do 
   ssh $i "
     for i in $(ls /dir/xyz_${i}*/details.txt); do 
        echo $i
     done
   "
done
not found: /dir/xyx_blue*/details.txt 
not found: /dir/xyx_green*/details.txt 
not found: /dir/xyx_red*/details.txt 

How can I get $i to expand when running a loop on the remote host?

0

2 Answers 2

1

The issue is that the command substitution that you want to run on the remote servers is run locally. This is due to the substitution occurring within double quotes, meaning the local shell will compute its expansion before calling ssh.

Also, reading the output of ls is ill-advised (see Why *not* parse `ls` (and what to do instead)?).

Instead:

servers=(
  blue
  green
  red
)

for server in "${servers[@]}"; do
    ssh -T "$server" <<END_SCRIPT
shopt -s nullglob
for pathname in /dir/xyz_$server*/details.txt; do
    printf 'file path is "%s"\n' "\$pathname"
done
END_SCRIPT
done

This uses an unquoted "here-document" as the script that is sent to the remote servers. The script first sets the nullglob shell option (this is assuming that the remote shell is bash) to avoid looping at all in the inner loop if the pattern does not match any filenames. It then iterates over the pattern /dir/xyz_$server*/details.txt where $server will be substituted with the current server name.

The loop then prints out a short message about what filenames matched the pattern, and the code ensures that the $pathname variable is not expanded by the local shell by escaping the $.

Note that $ in $server is unescaped and will therefore be expanded by the local shell, which is what we want, but that $ in $pathname needs to be escaped to supress the local expansion.

We don't need to set IFS to anything non-default. I'm assuming you did that to be able to write the array assignment as you did, or for some other reason, but none of the code above needs that.

I'm using -T with ssh to explicitly turn off pseudo-TTY allocation.


You could also provide the script as a single string if you feel that this is somehow needed:

servers=(
  blue
  green
  red
)

for server in "${servers[@]}"; do
    ssh "$server" "
shopt -s nullglob
for pathname in /dir/xyz_$server*/details.txt; do
    printf 'file path is \"%s\"\n' \"\$pathname\"
done"
done

In this case, you must additionally ensure that each embedded double quote is escaped (or they would end the double-quoted string that constitutes the remote shell script).

7
  • the ls example was stupid by me... I'm ssh'ing to various servers then parsing the multitude of 'red', 'green', 'blue' files. For that I need to capture output in variables using $(), but that doesn't work, also I'm having to escape variables for them to show like \$var and I'm using functions defined locally, I'll try your answer but thought I'd leave this question if you answer applies to my case.
    – Nickotine
    Jan 30 at 12:31
  • I set IFS as so, so I don't need to quote everything (only split words on newline not spaces).
    – Nickotine
    Jan 30 at 12:33
  • how can I get command substitution to occur on the remote server? Escaping doesn't work.
    – Nickotine
    Jan 30 at 12:35
  • @Nickotine A command substitution would be left unexpanded locally if you escape its $, as in \$(some-command).
    – Kusalananda
    Jan 30 at 12:47
  • 1
    I think I should ask another question since I just figured out my real issue.
    – Nickotine
    Jan 30 at 12:51
0

Never for-loop over the output of ls; that's literally what * globbing is for, and your whole usage of ls is totally superfluous.

That solves the whole problem, honestly, because you're currently running the $(…) locally instead of on the target host.

2
  • thanks but I have the same issue. The variable from the first loop is not getting expanded in the second loop.
    – Nickotine
    Jan 30 at 9:00
  • so my real issue is trying to get command substitution to occur on the remote, in the loop I have many $() saved to variables.
    – Nickotine
    Jan 30 at 12:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .