0

I want to use SFTP to upload files from A to B.

path for A: /tmp/A path for B: /tmp/B

server B has own username & password. my cronjob will run every 10 mins to check whether there are new files under /tmp/A, if yes then they will transfer the file, if not, no transfer.

  • 3
    Use rsync for that, not sftp. That's what rsync is for. Set up ssh keys with empty passwords for it, and set command="/usr/bin/rsync --server --daemon ." (or equivalent) for said keys on the receiving side. – Satō Katsura Jun 28 '16 at 3:28
1

You can use rsync command for this kind of operations.

How to use rsync to sync with a remote system

Syncing to a remote system is trivial if you have SSH access to the remote machine and rsync installed on both sides. You need to set up SSH keys, it's well described how to do it here: https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2

Once you have SSH access verified on between the two machines, you can sync the dir1 folder from earlier to a remote computer by using this syntax (note that we want to transfer the actual directory in this case, so we omit the trailing slash):

rsync -avz ~/dir1 username@remote_host:destination_directory

This is called a "push" operation because it pushes a directory from the local system to a remote system.

0

If I am understanding correctly I have found that some SFTP servers don't support ssh keys and therefore rsync won't work (reports.paypal.com as an example). In that case you'll need to script doing the sftp thing, you can use expect for this... Here's a script that should work for you:

#!/bin/sh

bname=`basename $0`

usage () #usage instructions
{
  cat <<EOF
  syncs LOCALPATH to REMOTEPATH on sftp://SERVICE

  Usage : ./$bname [-s SERVICE -l LOCALPATH -r REMOTEPATH] [-h]
    -h  display this help text.

  Example: ./$bname -s your.sftpserver.com -l /tmp/A -r /tmp/B
            will upload any missing files from /tmp/A to /tmp/B

EOF
  exit 0
}

upload ()
{
  security find-generic-password -s _$SERVICE_sftp >/dev/null 2>/dev/null #test if entry exists in keychain
  if [ $? -ne 0 ]; then
    echo "Please enter username for sftp://"$SERVICE
    read username
    echo $username > .credentials_tmp
    read -p "Password:" -s pass
    echo $pass >> .credentials_tmp
  else
    echo `security find-generic-password -s _$SERVICE_sftp | grep "acct" | cut -d \" -f 4` > .credentials_tmp
    echo `security find-generic-password -s _$SERVICE_sftp -w` >> .credentials_tmp
  fi
  echo $SERVICE >> .credentials_tmp
  echo $REMOTEPATH >> .credentials_tmp
  echo $LOCALPATH >> .credentials_tmp
  ls -1 $LOCALPATH > .LocalFileList.txt
  echo "
Connecting to sftp://"$SERVICE
  expect << 'EOS'
  log_user 0
  set send_human {.1 .3 1 .05 2}
  spawn sftp [exec sed "1p;d" .credentials_tmp]@[exec sed "3p;d" .credentials_tmp]:[exec sed "4p;d" .credentials_tmp]
  set timeout 30
  expect {
    "Password:" {
    # exp_continue
    } timeout {
    send_user "
  *** Connection timed out. Check connection and verify the username/password"; exit 1
    }
  }
  puts "Authenticating"
  send "[exec sed "2p;d" .credentials_tmp]\n"
  #log_user 1
  expect {
    "sftp>" {
    # exp_continue
    } timeout {
    send_user "
  *** Connection timed out. Check connection"; exit 1
    }
  }
  puts "Getting Remote File List"
  log_file -a -noappend .RemoteFileList.txt
  send "ls -1\n"
  expect {
    "sftp>" {
    # exp_continue
    } timeout {
    send_user "
  *** Connection timed out. Check connection"; exit 1
    }
  }
  log_file
  system sed -i '' '/ls -1/d' ./.RemoteFileList.txt
  system sed -i '' '/sftp>/d' ./.RemoteFileList.txt
  puts "Comparing with files in [exec sed "5p;d" .credentials_tmp]"
  sleep 1
  set rc [catch {exec diff -aw --changed-group-format=%< --unchanged-group-format= .LocalFileList.txt .RemoteFileList.txt} output]
  if {$rc == 0} {
    puts "no difference"
  } else {
    if {[lindex $::errorCode 0] eq "CHILDSTATUS"} {
      if {[lindex $::errorCode 2] == 1} {
        # send output without "child process exited abnormally" message
        set filename ".newfiles.txt"
        set fileId [open $filename "w"]
        puts -nonewline $fileId [string replace $output end-31 end ""]
        close $fileId
      } else {
        puts "diff error: $output"
      }
    } else {
      puts "error calling diff: $output"
    }
  }
  sleep 1
  if {[file size .newfiles.txt] == 0} {
    sleep 1
    send "bye\n"
    puts "
There are no local files missing from the remote server"
    interact
  } else {
    set f [open ".newfiles.txt"]
    while {[gets $f line] != -1} {
      send "put [exec sed "5p;d" .credentials_tmp]/$line\n"
      puts "Uploading $line..."
      expect {
        "sftp>" {
        # exp_continue
        } timeout {
        send_user "
      *** Connection timed out. Check connection"; exit 1
        }
      }
  }
  close $f
  send "bye\r"
  puts "
Finished syncing from [exec sed "5p;d" .credentials_tmp] to sftp://[exec sed "3p;d" .credentials_tmp]:[exec sed "4p;d" .credentials_tmp]"
  }
EOS
security find-generic-password -s _$SERVICE_sftp >/dev/null 2>/dev/null #test if entry exists in osx keychain
if [ $? -ne 0 ]; then
  while true; do
  read -p "
Do you want to save these login details for next time?
y/n " yn
    case $yn in
      [Yy]* )
        security add-generic-password -s _$SERVICE_sftp -a `sed '1p;d' .credentials_tmp` -w `sed '2p;d' .credentials_tmp` -U
        rm -f .credentials_tmp
        echo "credentials saved."
        break
        ;;
      [Nn]* )
        rm -f .credentials_tmp
        exit 1
        ;;
      * ) echo "Please answer yes or no.
y/n ";;
    esac
  done
fi
rm -f .credentials_tmp
rm -f .newfiles.txt
rm -f .LocalFileList.txt
rm -f .RemoteFileList.txt
}

# From here down is all to do with handling the flags you type into the command line, after the name of the script.
if [ ! $# == 6 ] || [ "$1" = "-h" ] || [ "$1" = "-" ]
then
  usage
fi

while getopts ":s:l:r:" opt; do
  case $opt in
    s) SERVICE=$OPTARG;;
    l) LOCALPATH=$OPTARG;;
    r) REMOTEPATH=$OPTARG;;
    \?)
      echo "Invalid option: -$OPTARG
      type ./"$bname" -h for help
      " >&2
      exit 1
      ;;
  esac
  #shift $(( OPTIND - 1 ))
done
upload $SERVICE $LOCALPATH $REMOTEPATH

To use, just save the code into a file e.g. uploadsftp.sh and run chmod u+x uploadsftp.sh

Then you can set cron to cd into the script directory and run the script: ./uploadsftp.sh -s SERVICE -l LOCALPATH -r REMOTEPATH whenever you need it to.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.