After ssh to remote hosts, grep used inside EOF gives output "No such file or directory"

Question

Scenario:

I have a script to connect to remote hosts and grep specific keyword from a log file. Without using EOF the ssh was getting freezed but when used heredoc (EOF) that issue was gone. All my commands have to be executed after ssh to remote host and hence they are within the EOF block.

grep command is giving me output as : 'No such file or directory' whereas I executed the same command on terminal and it gives desired output.

Below is the piece of code with problematic grep:

ssh -q -T username@remotehost <<-EOF 2>&1 | tee -a
getValue=$(grep -R 'keyword' /user/log/file.log | awk -F ' ' '{print$NF}') )
echo "${getValue}"
EOF

Output: grep: /user/log/file.log : No such file or directory

SSH is working fine as I printed the remote host and further cross checked the log path and file.log. From setup perspective everything is in place. Did some R&D over internet but nothing helped so far.

Answer 1

You need to quote your 'EOF' to prevent your shell from interpreting $ locally, ie. trying to run grep within your shell before the remote command.

Compare:

ssh username@remotehost << 'EOF'
echo "THIS IS $(hostname)"
EOF

with

ssh username@remotehost << EOF
echo "THIS IS $(hostname)"
EOF

Answer 2

Your issue comes from having the shell expand the command substitution in the here-document locally rather than on the remote system. It does this because the here-document is not quoted. It would additionally expand both $NF and ${getValue} as shell variables to empty strings for the same reason (unless these shell variables had values previous). The shell would not notice that $NF is used within an awk program.

Rather than trying to run complex commands on the remote host via ssh, just use ssh to get the data, then run the commands locally on that data:

ssh username@remotehost cat /user/log/file.log |
awk -F ' ' '/keyword/ { print $NF }'

I've simplified your pipeline somewhat too, combining that recursive (?!) grep with awk so that the awk command prints the last space-delimited field on the lines that contain the substring keyword.

Also note that there is no point to assigning output to a variable if all you want to do is to output the contents of that variable (echo may additionally modify the data that its given before printing under some circumstances). I've also removed the tee -a call that did not seem to do very much.

Answer 3

I achieved what I needed using wrapper script concept almost a week back. Just posting this ans for any new user who may struggle with same problem.

Surely, it's not a new concept and like many others I also knew about it before posting my question here. I really wanted to get this through a single script but somehow the variable expansion after ssh to remote hosts is not working with everything in same script. So it's not worth it spending so much of efforts.

Create a wrapper script and call your main script in it. Execute ssh comand in your wrapper passing the main script as argument. Something like:

ssh username@host < main_script.sh

In case you have multiple host, put above ssh in a for/while loop after reading the host server values in variable 'host'.

while read -r host
do
servers=${host}

ssh username@${host} < main_script.sh

done < ${servers}

Thanks everyone for pitching in with your suggestions!!