Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am developing software that will utilize inotify to track changes on a large amount of files (tens to hundreds of thousands of files). I have come up with these ideas:

  • one watch per file
  • one watch per parent directory
  • avoid inotify and periodically scan the fs for changes (not preferred)

I will have a database of all of the files I am watching and some basic stat information (like mtime and size), however, I would have to stat each file in that directory until I found the one that changed.

Which would be faster, tons (100,000+) of inotify watches or tons of stat calls?

I'm thinking that reducing the number of stat calls would be better, but I don't know enough about inotify.

Note:

This will be running on a workstation, not a server. It's main purpose is to synchronize changes (potentially to an entire filesystem) between a client and a remote server.

share|improve this question
    
There are filesystems designed for this kind of replication. I recommend investigating them rather than reinventing the locomotive (it's more complex than a wheel). I don't have a filesystem to offer but I remember one being mentioned on this site recently, hopefully someone can chime in with proposals. –  Gilles Aug 10 '11 at 0:55
    
I would be very interested. The only thing I can think of is an NFS filesystem, which isn't exactly what I'm looking for. –  tjameson Aug 10 '11 at 4:59
1  
Ah, found it: ChironFS. Even if you don't end up using it, its design documentation and source code are likely to be interesting reads. –  Gilles Aug 10 '11 at 7:26
    
@Gilles- Thanks. I assumed it would be a FUSE-based fs. I'll probably end up needing more features than it has, and it looks a little too young, but definitely an interesting read. –  tjameson Aug 10 '11 at 15:37
add comment

1 Answer 1

up vote 2 down vote accepted

When you read() an inotify fd, the name field of the returned struct tells you which file was modified relative to the directory being watched, so you shouldn't have to stat every file in a directory after the event.

See http://linux.die.net/man/7/inotify

Specifically:

struct inotify_event {
    int      wd;       /* Watch descriptor */
    uint32_t mask;     /* Mask of events */
    uint32_t cookie;   /* Unique cookie associating related
                          events (for rename(2)) */
    uint32_t len;      /* Size of 'name' field */
    char     name[];   /* Optional null-terminated name */
};

The name field is only present when an event is returned for a file inside a watched directory; it identifies the file pathname relative to the watched directory. This pathname is null-terminated, and may include further null bytes to align subsequent reads to a suitable address boundary.

share|improve this answer
    
So, is there any gain to adding a listener to individual files? –  tjameson Aug 9 '11 at 1:50
    
Only if there are going to be many events on files you don't care about in that directory. –  Shawn J. Goff Aug 9 '11 at 1:55
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.