34

There is a directory A whose contents are changed frequently by other people.

I have made a personal directory B where I keep all the files that have ever been in A.

Currently, I just occasionally run rsync to get the files to be backed up from A to B. However, I fear the possibility that some files will get added in A, and then removed from A before I get the chance to copy them over to B.

What is the best way to prevent this from occurring? Ideally, I'd like to have my current backup script run every time the contents of A get changed.

4 Answers 4

40

If you have inotify-tools installed you can use inotifywait to trigger an action if a file or directory is written to:

#!/bin/sh
dir1=/path/to/A/
while inotifywait -qqre "attrib,modify,close_write,move,move_self,create,delete,delete_self" "$dir1"; do
    /run/backup/to/B 
done

Where the -qq switch is completely silent, -r is recursive (if needed) and -e is the event to monitor. From man inotifywait:

attrib The metadata of a watched file or a file within a watched directory was modified. This includes timestamps, file permissions, extended attributes etc.

modify A watched file or a file within a watched directory was written to.

close_write A watched file or a file within a watched directory was closed, after being opened in writeable mode. This does not necessarily imply the file was written to.

move A file or directory was moved from or to a watched directory. Note that this is actually implemented simply by listening for both moved_to and moved_from, hence all close events received will be output as one or both of these, not MOVE.

move_self A watched file or directory was moved. After this event, the file or directory is no longer being watched.

create A file or directory was created within a watched directory.

delete A file or directory within a watched directory was deleted.

delete_self A watched file or directory was deleted. After this event the file or directory is no longer being watched. Note that this event can occur even if it is not explicitly being listened for.

6
  • 7
    incrond may also be of interest. Commented Nov 6, 2011 at 13:50
  • 1
    Cool. Would I just put the above script in my bashrc so that it run's when i log in? Is there another way I can have it running all the time?
    – oadams
    Commented Nov 7, 2011 at 11:53
  • For login, yes /etc/profile for system-wide or .bash_profile for just your user. To run it after boot, it depends on your flavour of Unix/Linux; /etc/rc.local,/etc/rc.d/ or /etc/init.d/
    – jasonwryan
    Commented Nov 7, 2011 at 17:05
  • 3
    For larger directories you might want to consider using the --monitor switch (and pipe the output to your loop instead), otherwise there is a lot of overhead when inotifywait is started over and over again Commented Jan 10, 2013 at 17:10
  • 2
    I find that close_write is a more reliable event to use when monitoring changes. modify could be called many time for the same file, close_write is called once when the file is closed after writing.
    – datashaman
    Commented Jan 26, 2020 at 16:18
14

Try entr command-line tool which can run arbitrary commands when files change. Since 2.9 release, a directory watch option (-d) was added to react to events when a new file is added to a directory.

Example to run the utility if a new file is added to the project:

$ while true; do
> echo src/* | entr -d your_command
> done

In directory watch mode the parent directory of each file is implicitly added to the watch list.

The only implication of this is that if a new file appears it must exit to allow an external shell loop to rescan the file system.

Here is the version without a directory watch option:

$ while true; do
> echo src/* src | entr your_command
> done

Here is a simpler example depending on your needs:

$ ls -d * | entr sh -c 'rsync -vuar A B'

Check: eradman.com/entrproject website for more details.

5

Strictly speaking, if someone drops a file and very quickly removes it, you might miss it. The use of inotify (under Linux, or a similar feature under other unices) makes the window of risk small.

If you can mount the filesystem of your choice on that directory (I realize this may not be an option), you can put one that records all file versions, for example copyfs.

1
4

The program you're looking for is inotify.

2
  • Linux-only though Commented Aug 15, 2020 at 11:44
  • inotify is not a command, inotifywait is
    – Flimm
    Commented Jan 25, 2023 at 15:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .