7

We have a server where another script sFTPs and downloads files everyday.

Question: Is it possible to detect that the file was downloaded and then for me to automatically archive the file after they are done?

To clarify - We host the files and someone else comes and downloads it.

This is the script they use:

let $command = 'sftp -b /usr/tmp/file.sftp someuser@myserver'
show 'FTP command is ' $command
call system using $command #status

##file.sftp##

# Set local directory on PeopleSoft server
lcd /var/tmp
# Set remote directory on the remote server
cd ar/in
# Transfer all remote files to PeopleSoft
get file.dat
get file2.dat
# quit the session
bye
2
  • Are the files downloaded via a cron job
    – eyoung100
    Commented Dec 2, 2014 at 22:03
  • 3
    Is there a reason not to modify the original script to archive things when it is done?
    – Suchipi
    Commented Dec 3, 2014 at 0:11

3 Answers 3

13

There are 3 avenues that I can conceive of that might provide you with a solution.

1. Custom sftp Subsystem

You could wrap the sftp-server daemon via sshd_config and "override" it with your own script that could then intercept what sftp-server is doing, and then act when you see that a file was downloaded. Overriding the default sftp-server in sshd_config is easy:

Subsystem       sftp    /usr/local/bin/sftp-server

Figuring out what to do in the wrapper script would be the hard part.In /usr/local/bin/sftp-server:

#!/bin/sh

# ...do something...
chroot /my/secret/stuff /usr/libexec/openssh/sftp-server
# ...do something...

2. Watch the logs

If you turn up the debugging of sftp-sever you can get it to show logs of when files are being open/closed and read/written to/from the SFTP server. You could write a daemon/script that watches these logs and then backs the file up when needed. Further details on how to achieve this are already partially covered in my answer to this U&L Q&A tiled: Activity Logging Level in SFTP as well as here in this blog post titled: SFTP file transfer session activity logging.

The SFTP logs can be enhanced so they look like this:

Sep 16 16:07:19 localhost sftpd-wrapper[4471]: user sftp1 session start from 172.16.221.1
Sep 16 16:07:19 localhost sftp-server[4472]: session opened for local user sftp1 from [172.16.221.1]
Sep 16 16:07:40 localhost sftp-server[4472]: opendir "/home/sftp1"
Sep 16 16:07:40 localhost sftp-server[4472]: closedir "/home/sftp1"
Sep 16 16:07:46 localhost sftp-server[4472]: open "/home/sftp1/transactions.xml" flags WRITE,CREATE,TRUNCATE mode 0644
Sep 16 16:07:51 localhost sftp-server[4472]: close "/home/sftp1/transactions.xml" bytes read 0 written 192062308
Sep 16 16:07:54 localhost sftp-server[4472]: session closed for local user sftp1 from [172.16.221.1]

You would then need to develop a daemon/script that would monitor the logs for the open/close event pairs. These represent a completed file transfer. You could also make use of syslog, which could monitor for the "CLOSE" log events and it could be used to perform the copying of the transferred files.

3. Incron

You could make use of Inotify events that the Linux kernel produces every time a file is accessed. There is a service called Incron which works similarly to Cron. Where Cron works based on time, Incron works based on file events. So you could setup a Incron entry that would monitor your SFTP upload directories, and any time a specific file event is detected, copy the file.

Have a look at the inotify man page for description of the various events. I believe you'd want to watch for a read() (IN_ACCESS) followed by a close() (IN_CLOSE_WRITE). These would be for files that were copied from the SFTP server.

Incron rules look like this:

<directory> <file change mask> <command or action>  options
/var/www/html IN_CREATE /root/scripts/backup.sh
/sales IN_DELETE /root/scripts/sync.sh
/var/named/chroot/var/master IN_CREATE,IN_ATTRIB,IN_MODIFY /sbin/rndc reload

This article titled: Linux incrond inotify: Monitor Directories For Changes And Take Action shows much more of the details needed, if you want to try and go with this option.

3

Configure the SSH server to log the activity, then you can parse the log to know if such file has been downloaded.

To enable logging append -l INFO to the sftp subsystem line in the /etc/ssh/sshd_config file, it should look something like (the path may vary by distro, I'm using SuSE 11):

Subsystem    sftp    /usr/lib64/ssh/sftp-server -l INFO

Now the sftp sessions will be logged in /var/log/messages.

This is a log for file downloaded by sftp:

Dec  3 08:42:02 $HOSTNAME sftp-server[$PID]: session opened for local user user from [192.168.0.10]
Dec  3 08:42:03 $HOSTNAME sftp-server[$PID]: opendir "/home/user"
Dec  3 08:42:03 $HOSTNAME sftp-server[$PID]: closedir "/home/user"
Dec  3 08:42:18 $HOSTNAME sftp-server[$PID]: open "/home/user/file" flags READ mode 0666
Dec  3 08:42:19 $HOSTNAME sftp-server[$PID]: close "/home/user/file" bytes read 48843 written 0

Now you can parse the file to know if the file has been accessed with something like:

#!/bin/bash

grep sftp /var/log/messages | grep -q $FILENAME
if [ $? -eq 0 ]
    do something
else
    do other thing
fi

How to enable SFTP logging: https://serverfault.com/questions/73319/sftp-logging-is-there-a-way

1
  • 1
    Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference. Also I realize it's another SE site, and I wrote one of the A'ers, but we still like content to be free standing, if possible.
    – slm
    Commented Dec 3, 2014 at 1:03
-1

If you know the path and name then you can do this

if [ -f $filename ];
then 
archive
else
echo ""
fi

However you did not add any such information.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .