0

Scenario:

I have a script to connect to remote hosts and grep specific keyword from a log file. Without using EOF the ssh was getting freezed but when used heredoc (EOF) that issue was gone. All my commands have to be executed after ssh to remote host and hence they are within the EOF block.

grep command is giving me output as : 'No such file or directory' whereas I executed the same command on terminal and it gives desired output.

Below is the piece of code with problematic grep:

ssh -q -T username@remotehost <<-EOF 2>&1 | tee -a
getValue=$(grep -R 'keyword' /user/log/file.log | awk -F ' ' '{print$NF}') )
echo "${getValue}"
EOF

Output: grep: /user/log/file.log : No such file or directory

SSH is working fine as I printed the remote host and further cross checked the log path and file.log. From setup perspective everything is in place. Did some R&D over internet but nothing helped so far.

3 Answers 3

0

You need to quote your 'EOF' to prevent your shell from interpreting $ locally, ie. trying to run grep within your shell before the remote command.

Compare:

ssh username@remotehost << 'EOF'
echo "THIS IS $(hostname)"
EOF

with

ssh username@remotehost << EOF
echo "THIS IS $(hostname)"
EOF
0

Your issue comes from having the shell expand the command substitution in the here-document locally rather than on the remote system. It does this because the here-document is not quoted. It would additionally expand both $NF and ${getValue} as shell variables to empty strings for the same reason (unless these shell variables had values previous). The shell would not notice that $NF is used within an awk program.

Rather than trying to run complex commands on the remote host via ssh, just use ssh to get the data, then run the commands locally on that data:

ssh username@remotehost cat /user/log/file.log |
awk -F ' ' '/keyword/ { print $NF }'

I've simplified your pipeline somewhat too, combining that recursive (?!) grep with awk so that the awk command prints the last space-delimited field on the lines that contain the substring keyword.

Also note that there is no point to assigning output to a variable if all you want to do is to output the contents of that variable (echo may additionally modify the data that its given before printing under some circumstances). I've also removed the tee -a call that did not seem to do very much.

7
  • I think OP gave us a minimal working example. and reals OP's shell is more complex, echo ${GetValue} would be show as a residual token, while actual shell might use it.
    – Archemar
    Jul 22, 2021 at 14:03
  • @Archemar There is no indication that the code in the question is a minimal working example. In fact, the text explicitly says "Below is the piece of code with problematic grep", which makes me assume that this is the actual code they are using (the inclusion of tee -a also would not be part of a minimal working example). In any case, there is no other code in the question so this is all I have to go on. If the user needs to save the output of the awk in a variable, it would be easy to put my pipeline in a command substitution.
    – Kusalananda
    Jul 22, 2021 at 14:48
  • @Kusalananda - i have a lot of code which needs to execute on multiple remote hosts. Part of business requirement. Have achieved executing commands on remote hosts using <<-EOF in one of other scripts. Not sure why the hell this one has issues. Anyways script runs in a loop to cover all the remote hosts to grep keyword in their respective log directories. What I have posted is just the first line of code after ssh where it fails. What remains after this are too many validations to be done. This also rules out the solution to use ssh to get the data, then run the commands locally on that data.
    – MayD
    Jul 22, 2021 at 17:38
  • @user15762223 I don't quite see why the validations need to happen remotely though. That just complicates the code. In any case, as both I and the other person have noted, the issue is due to not quoting the here-document. You would do that by using <<'EOF' to start the here-document.
    – Kusalananda
    Jul 22, 2021 at 17:44
  • @Kusalananda - using <<'EOF' fails script and returns syntax error - unexpected end of file. As I see it, quoting EOF disables all my code below it. It also does not allows variable expansion on remote hosts as I read somewhere. Tried this also << 'EOF' but no luck. As I mentioned, I wrote a different script for some other project requirement to connect to remote hosts and do some actions, it worked fine using ssh and <<-EOF. I just had to use $(declare -p variable) that were to be used on remote hosts. Tried using declare in this case, but strangely it's not working.
    – MayD
    Jul 22, 2021 at 18:05
0

I achieved what I needed using wrapper script concept almost a week back. Just posting this ans for any new user who may struggle with same problem.

Surely, it's not a new concept and like many others I also knew about it before posting my question here. I really wanted to get this through a single script but somehow the variable expansion after ssh to remote hosts is not working with everything in same script. So it's not worth it spending so much of efforts.

Create a wrapper script and call your main script in it. Execute ssh comand in your wrapper passing the main script as argument. Something like:

ssh username@host < main_script.sh

In case you have multiple host, put above ssh in a for/while loop after reading the host server values in variable 'host'.

while read -r host
do
servers=${host}

ssh username@${host} < main_script.sh

done < ${servers}

Thanks everyone for pitching in with your suggestions!!

1
  • Passing the script on the standard input of ssh is exactly the same as using a here-document as in your question. The only difference is that the here-document that you use in your question is unquoted, leading to the errors that you see. The other answers are pointing this out. It's further unclear why you set servers in your loop.
    – Kusalananda
    Aug 2, 2021 at 10:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .