3

I need to perform an automated backup task that connects to an SFTP server. The first stage involves testing the server connection by connecting to it. To be absolutely sure that it is connecting to the correct server, I am setting the StrictHostKeyChecking in ssh_config to yes.

So, when the connection to the SFTP server is made, how do I script the response from the server. The pseudo code is as follows:

If connection success
  disconnect
  initiate backup procedure

Else
  log error
  exit script

Additional Info:

I am using ForceCommand internal-sftp in the backup server's sshd_config, so running ssh command is out of the question.

  • I've provided a shell script in my answer that you can use the exit code from in the "connection success test". It includes the "disconnect" part. The only way I can think of to do the checking and other dependant logic branching inside the same sftp session is to script it with expect/tcl which never becomes elegant, as it is very hard to parse text and foresee any possible error conditions before you have found them the first time. Using for example lftp, the client itself checks the sftp-protocol's built-in status response codes. This makes catching errors easy. – MattBianco Aug 5 '14 at 8:48
1

Most "SSH File Transfer Protocol (SFTP) clients" are primarily/initially built for interactive use.

This doesn't mean that they lack batch mode options and such, but if you're looking for a script based file transfer over the SSH protocol, have a look at scp or lftp instead.

lftp (homepage) is what I use in all my automated file transfers, and it has excellent error handling with reliable exit status. It was designed with reliability in mind.

There are tons of options to lftp, and you can supply any ssh options you like, such as -oStrictHostKeyChecking=yes if you specify a specific connection program to use.

Example:

MYSSHOPTIONS="-oStrichtHostKeyChecking=yes"
CONNPROG="ssh -a -x ${MYSSHOPTIONS}"

cat > ${cmdfile}  <<- EOF
set sftp:max-packets-in-flight 16
set sftp:connect-program ${CONNPROG}
open -u ${remoteuser},${password-unless-pubkey-setup} sftp://${remotehost}
put localfile -o remotepathname
EOF

lftp -f ${cmdfile} > ${sendlog} 2>&1
RC=$?
if test $RC -ne 0
then
   failed
else
   reliably OK
fi

The example is a bit long. It creates a command file that sets some options and uploads one file localfile with optionally another remotepathname on the other side.


sftp variant if you don't want to use lftp:

If you are interested in checking sftp login access you can use this template as a starting point for further experimenting:

#!/bin/sh -

mykey=/home/localuser/.ssh/id_rsa
remusr=bupuser
remhost=server.destination.domain.com
tmpfile=/tmp/sftptest.$$

cleanup() {
  rm -f ${tmpfile}
}
trap cleanup 0

sftp -i $mykey -oPubkeyAuthentication=yes -oPasswordAuthentication=no -oKbdInteractiveAuthentication=no -oStrichtHostKeyChecking=yes ${remusr}@${remhost}  ${tmpfile} 2>&1
  dir
  exit
EOF
ST=$?

if test $ST -ne 0
then
  echo SFTP LOGIN FAILURE. RC=${ST} 1>&2
  exit $ST
fi

cat ${tmpfile} # or do some clever grepping on it

exit $ST
  • Thank you. I think this may be useful though I was expecting something more straight forward. Seems like all answers here seems to suggest that it is not possible to obtain status from sftp directly. – Question Overflow Aug 4 '14 at 15:03
  • Well.. If you're mainly interested in validating that login is possible and that you are connecting to the correct server, scripting it with the normal sftp client from the openssh package is certainly possible if you have set up public key authentication and don't mind parsing stdout output for extra safety. If you want it robust and easy to expand I recommend lftp though. It is very easy and has many features you might want later on. It's just my example that is not so pretty. – MattBianco Aug 5 '14 at 8:26
  • Thank you. I think I will adopt your method and pipe stdout to dev/null so that cron will not send me email every time. – Question Overflow Aug 6 '14 at 7:22
2

I've written expect and bash scripts to work this.... (though it can get fancier as mentioned above).

Bash script: executed with three arguments: user, pass, and host. For example ./bwrap.sh ninja_user ninja_star1234 ninja.com

The script creates a log file which later used for checking successful log in/out.

#!/bin/bash

# log file
log="connection_test.log"
if [ -f $log ];
then
    echo "Older file $log exists, removing and creating new..."
    rm -rf $log
    touch $log
else
    echo "Creating log file"
    touch $log
fi

# running expect script.
# arg 1 is user, arg 2 is pass, arg 3 is host 
./one.exp $1 $2 $3  >> $log

# checking log for connections 
if grep --quiet logout $log; then
    echo "connection successful, proceeding to backup" >> $log
else
    echo "connection failed, please check server" >> $log
    exit 1
fi

Now to the expect script, it has a timeout of 10 seconds, and I actually prefer to work w/o strict host key. If you do want to work with strict host key edit the relevant line and add an expect "yes/no" ....

#!/usr/bin/expect -f

set user [lindex $argv 0];
set password [lindex $argv 1];
set host [lindex $argv 2];

set timeout 10
# now ssh
spawn ssh $user@$host -o StrictHostKeyChecking=no
match_max 100000 # Look for passwod prompt
expect "*?assword:*"
# Send password aka $password
send -- "$password\r"

expect "*$ "
send -- "whoami\r"
expect "<user name>"  # change user
send -- "exit\r"
expect eof

Hope this helps.

For sftp connection please: add line to bash script where $4 is port

./one.exp $1 $2 $3 $4 >> $log

add line to expect script:

set port [lindex $argv 3];

replace in expect script

spawn ssh $user@$host -o StrictHostKeyChecking=no

with

spawn sftp $host@$user -o Port=$port StrictHostKeyChecking=no

and

 expect "*$ "

with

expect "sftp>"

Since I do not have an SFTP server to test it on, the changes are made based on [this question]: sftp a file using shell script

Also, [expect homepage]: http://expect.sourceforge.net/ might be handy.

Lastly, reading your comment regarding a straight forward approach, it's possible to simply use nc and see if the host:port is up. For this you can use:

#!/bin/bash

# log file
log="connection_test.log"
if [ -f $log ];
then
    echo "Older file $log exists, removing and creating new..."
    rm -rf $log
    touch $log
else
    echo "Creating log file"
    touch $log
fi

# nc
nc_command=`nc -z -w 5 $1 $2 | tee -a $log`
if [[ $nc_command == *succeeded* ]];
then
    echo "Server is listening, ready for backup" | tee -a $log
else
    echo " Server seems to be offline, please check" | tee -a $log
    exit 1
fi 

To run the last script use: ./test.sh host port

  • Thank you. Can this be modified to connect with sftp command instead of ssh? Have updated my question to clarify that point. – Question Overflow Aug 4 '14 at 14:58
  • @QuestionOverflow Yes, please review edits at the bottom. Note that I do not have sftp to connect to, so this is based from examples. – Simply_Me Aug 4 '14 at 15:10
  • @QuestionOverflow added lines needed to change to sftp and another solution using more straight forward approach. – Simply_Me Aug 4 '14 at 16:23
  • +1 for the effort. nc is new to me. I will try to learn how to use it. – Question Overflow Aug 6 '14 at 7:18
0

You could test sftp connectivity by just doing an ssh connection test:

if $(ssh -q belmin@mysever.bfworks.com exit)
then
    echo yes
fi

If the server does not allow ssh, you could do use nmap but that doesn't test key authentication:

nmap myserver.bfworks.com -PN -p ssh | grep 'open'
  • 2
    assuming that the SFTP server allows SSH login – MattBianco Aug 4 '14 at 13:45
  • Correct. I've used an nmap command line for servers that do not allow SSH login. I'll add that to the answer just in case it's useful. Thanks! – Belmin Fernandez Aug 4 '14 at 13:50
  • 2
    No, I mean the ssh shell subsystem. You can configure sshd to only allow (or force) sftp only. The user account can have a shell that only allows sftp, and so on. – MattBianco Aug 4 '14 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.