Script to get file from SFTP and run it

Question

We have 5 servers running our code: A, B , C, D and E.

From a local machine, we sftp the file to A. Then

ssh id@A
su - applicationname
sftp id@A
get <file>
bye
tar -xzf <file>
logout
logout

The same procedure we apply for others B to E.

Is there any way we can simplify this using script?

We have requested linux team to provide passwordless authentication. But we don't know how to do sftp inside a SSH session using a script.

Answer 1

Best and easiest is to use scp instead of sftp.

If scp is really not an option, you can read sftp commands using a 'here document', but it is very hard to script around errors.

sftp username@server << %EOF%
ls
bye
%EOF%

%EOF% can be any word you want, as long as it starts and ends with a % and the last %EOF% must be at the start of a line.

For passwordless login, read up on 'ssh key authentication'.

Using sftp from within an ssh session sounds like overkill, sounds like doing double things. Why not initiate the copy from the local machine?

Answer 2

You can use lftp to do password authentication from a script. Password-less authentication can be very dangerous sometimes. Since I see that you are invoking this script for multiple servers, you will have to manually enter the passwords every time you use sftp.

Instead, you can use lftp to put your password inside the script and run it as a cron-job. Although you will be needing different files for each server. Nevertheless, all of them can be run through a cron-job to run *.sh inside a directory that contains each copy of the script.

Steps to follow for password authentication from a script:

1. firstly install lftp using yum install lftp or sudo apt-get install lftp, according to the linux flavor.

2. write a script as below:

   #!/bin/sh
   
   HOST='ec2#####.us-west-2.compute.amazonaws.com'
   
   USER='ubuntu'
   
   PASSWD='ubuntuuser'
   
   cd directory/where/you/want/to/put/or/get/a/file
   
   lftp << END_OF_SCRIPT
   open sftp://$HOST
   user $USER $PASSWD
   cd /home/anywhere_you_want_the_file_from/
   mput *.txt
   END_OF_SCRIPT
   

---

Note, please change the hostname, username and password according to your server. Also this script uses 'mput' to get multiple files at /home/anywhere_you_want_the_file_from/ location at remote location to the local. If single file is needed 'put' can be used. Any commands of sftp can be used inside the END_OF_SCRIPT.