Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a program that is running on multiple machines and which is writing the results to ~/data/machine_name/, each time the results file exceeds 1MB the program moves on and write to a new file. (All machines have the same directory architecture).

I want to do an automatic backup of these files in case one of the machines fail. Knowing that the most recent accessed file is the one being written by the program, and hence, I would copy all the old files except this one. The next time the backup job fires, the most recent file omitted in the previous backup will be included and so on..

Right now I am using only a simple scp command, which copies even the file being open by the remote machine.

Is there anyway to achieve this? a command line utility or a script for such a job?

share|improve this question
Are your files opened for writing and continuously kept open until full or do they get open and closed a lot and written line by line? – Caleb Jul 27 '11 at 19:50
As far as a file is not Full, it still opened till it gets full, once full it's closed – 0xFF Aug 2 '11 at 8:04
up vote 2 down vote accepted

You can use rsyncbackup utility with --exclude option:

/usr/bin/rsync -avz --exclude='$EXCLUDED_DIR_OR_FILE' -e ssh /your/base/folder \
user@host_or_ip:/target/machine/path >> /optional/log/file.log

You can determine $EXCLUDED_DIR_OR_FILE variable by finding last updated file. There are several ways to find last updated file in a given directory, one of them is using ls and awk utilities together:

ls -lrt | awk '{ f=$NF }; END{ print f }'

Please note that this command prints last modified file name and not he full path to the file.

share|improve this answer
Parsing the output of ls is to be avoided. – Caleb Jul 27 '11 at 19:45
@Caleb I just gave it as an example hint. – Emre Yazici Jul 27 '11 at 20:06

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.