24

There is a directory A whose contents are changed frequently by other people.

I have made a personal directory B where I keep all the files that have ever been in A.

Currently, I just occasionally run rsync to get the files to be backed up from A to B. However, I fear the possibility that some files will get added in A, and then removed from A before I get the chance to copy them over to B.

What is the best way to prevent this from occurring? Ideally, I'd like to have my current backup script run every time the contents of A get changed.

26

If you have inotify-tools installed you can use inotifywait to trigger an action if a file or directory is written to:

#!/bin/sh
dir1=/path/to/A/
while inotifywait -qqre modify "$dir1"; do
    /run/backup/to/B 
done

Where the -qq switch is completely silent, -r is recursive (if needed) and -e is the event to monitor, in this case modify. From man inotifywait:

modify
A watched file or a file within a watched directory was written to.
  • 5
    incrond may also be of interest. – Shawn J. Goff Nov 6 '11 at 13:50
  • 1
    Cool. Would I just put the above script in my bashrc so that it run's when i log in? Is there another way I can have it running all the time? – oadams Nov 7 '11 at 11:53
  • For login, yes /etc/profile for system-wide or .bash_profile for just your user. To run it after boot, it depends on your flavour of Unix/Linux; /etc/rc.local,/etc/rc.d/ or /etc/init.d/ – jasonwryan Nov 7 '11 at 17:05
  • 1
    For larger directories you might want to consider using the --monitor switch (and pipe the output to your loop instead), otherwise there is a lot of overhead when inotifywait is started over and over again – Tobias Kienzler Jan 10 '13 at 17:10
  • How reliably would this solution work to run a script for each file that was added in a directory tree through an FTP server (let's say that a new file is added every 3 minutes on average and this might go on for years)? If not very, is there an alternative solution? – Nikola Novak Sep 10 '15 at 8:50
5

Strictly speaking, if someone drops a file and very quickly removes it, you might miss it. The use of inotify (under Linux, or a similar feature under other unices) makes the window of risk small.

If you can mount the filesystem of your choice on that directory (I realize this may not be an option), you can put one that records all file versions, for example copyfs.

4

Try entr command-line tool which can run arbitrary commands when files change. Since 2.9 release, a directory watch option (-d) was added to react to events when a new file is added to a directory.

Example to run the utility if a new file is added to the project:

$ while true; do
> echo src/* | entr -d your_command
> done

In directory watch mode the parent directory of each file is implicitly added to the watch list.

The only implication of this is that if a new file appears it must exit to allow an external shell loop to rescan the file system.

Here is the version without a directory watch option:

$ while true; do
> echo src/* src | entr your_command
> done

Here is a simpler example depending on your needs:

$ ls -d * | entr sh -c 'rsync -vuar A B'

Check: entrproject.org website for more details.

3

The program you're looking for is inotify.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.