Within a .sh script, I'm using expect to automatically sftp to another computer in order to get files. My script will be set to run every 30 minutes, at which point I want to get only files that were created (modified) since the last run (risking repetition, within the last 30 minutes). I know that the files will follow a specific naming method: AppData*.RAW where * will be a date and time string. However, I do not know when the files will be created or transferred to the computer I am communicating with, nor do I know how many there could be as the number will vary.

spawn sftp user@host
expect "User's password: "
send "$password\r"
expect "sftp>"
send "get ????"

I searched for help with get and found references to using the -neweronly switch, as well as trying synchronize. However, I don't know how to implement either of these, or if synchronize is even an option in sftp dialogue. Help with get

Also, the computer that I have to sftp to stores incoming files in a directory with thousands of older files (from 2013 to present), so it's imperative that I only get files from the last half hour.

  • 1
    Would rsync fit your needs better? – jordanm Sep 16 '15 at 22:39
  • Do you even know ssh key pair ? Expect is not a good practice – Gilles Quenot Sep 16 '15 at 22:41
  • @jordanm I don't know if rsync is something I could use. Do both computers need to have it? I think the other comp is a Windows machine, but the admin hasn't been overly helpful. @Gilles I have tried using the ssh key pair, but it does not work. I cant actually ssh to the other machine, only sftp via PuTTy or FileZilla. – Alex26 Sep 16 '15 at 22:47
  • @jordanm As a follow up, I tried using rsync, but I get "exec request failed on channel 0," which is the same as when I try to just ssh to it. – Alex26 Sep 16 '15 at 23:05
  • You referred to documentation for winscp. Is that the client you're using, or are you using the sftp from the openssh package? – Mark Plotnick Sep 17 '15 at 11:24

sftp has limited capabilities. The easiest way to do complex things over SFTP is to use SSHFS. SSHFS is a filesystem that uses SFTP to make a remote filesystem appear as a local filessytem. On the client, SSHFS requires FUSE, which is available on most modern unices. On the server, SSHFS requires SFTP; if the server allows SFTP then you can use SSHFS with it.

For example, to copy the files modified in the past 30 minutes, first mount the remote directory over SFTP with sshfs. Use sshpass if you have to use a password for SSH. Then use the command you'd use locally (here, I use GNU utilities, but depending on what you're doing, rsync -a mnt/ /local/path/ might be better). Unmount the SSHFS filesystem when you've finished.

mkdir mnt
sshpass -f password.txt sshfs user@host: mnt
find mnt -mmin -30 -exec cp -t /local/path/ {} +
fusermount -u mnt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.