Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

How do I detect new files in a folder with a bash script? I would like to process the files as soon as they are created in the folder. Is this possible to do so or do I have to schedule a script with cron that that check for new files each minute or so?

share|improve this question
    
Are you going to remove files from the folder once they are processed? –  ztank1013 Nov 19 '11 at 20:00
    
possible duplicate of How to run a command when a directory's contents are updated? –  Gilles Nov 19 '11 at 23:35
add comment

5 Answers

up vote 17 down vote accepted

You should consider using inotifywait, as an example:

inotifywait -m /path -e create -e move_to |
    while read path action file; do
        echo "The file '$file' appeared in directory '$path' via '$action'"
        # do something with the file
    done

In Ubuntu inotifywait is provided by the inotify-tools package. As of version 3.13 (current in Ubuntu 12.04) inotifywait will include the filename without the -f option. Older versions may need to be coerced. What is important to note is that the -e option to inotifywait is the best way to do event filtering. Also, your read command can assign the positional output into multiple variables that you can choose to use or ignore. There is no need to use grep/sed/awk to preprocess the output.

share|improve this answer
    
Great! The inotifywait was just what I wanted. –  ihatetoregister Nov 19 '11 at 18:50
    
This is a nice utility, thanks! –  ztank1013 Nov 19 '11 at 21:07
    
Why is fflush() necessary here? Since a newline is already created I think –  warl0ck Oct 18 '12 at 2:52
    
@warl0ck: I think awk does not print each line as it can, but it waits for a bigger buffer (containing more than one line) before it decides to output something. –  enzotib Oct 18 '12 at 17:12
1  
Just want to update this. You do not need awk to achieve this. you can filter the events with '-e create' and get only the filename by doing '-f %f' or the full path using '-f %w%f'. So the first line of the above script becomes: inotifywait -m /path -f %w%f -e create | –  Lugoues Mar 3 '13 at 16:44
show 1 more comment

I prefer incron, as its easier to manage. Essentially its a service that leverages inotify and you can setup configurations to take action based on file change operations.

Ex:

<directory> <file change mask> <command or action>  options
/var/www/html IN_CREATE /root/scripts/backup.sh

You can see a full example here: http://www.cyberciti.biz/faq/linux-inotify-examples-to-replicate-directories/

share|improve this answer
add comment

Bash cannot do this easily. You'd have to basically get a list of all the files in the folder and periodically get a new list and compare them to see whats changed.

What you're looking for is called inotify. Its built into the linux kernel and you can basically sit there waiting for something to happen at which point inotify comes back and says 'hey, theres a new file called foobar'

To accomplish what you want you'd have to switch to something like perl and use Linux::Inotify2 (python probably supports inotify as well, but I'm a perl person).

share|improve this answer
add comment

I am assuming the target folder (I'll call it isempty just for convenience) is empty and you are waiting for one or more files to be dropped there.

You can use the following command:

ls -1A isempty | wc -l

just to check if the folder is still empty, in fact it will return a 0 if there is no new file (hence the isempty folder is still empty) or, on the other hand, it will return a value greater than 0 (actually the number of files currently in the folder).

That said a silly if/then test can make the rest of the work:

if [ $(ls -1A isempty | wc -l) -gt 0 ] ; then do_something ; fi

Of course the do_something function will have to manipulate the file(s) within the isempty folder and then remove it(them) from the folder itself after processing.

Adding a line like the following in your crontab will run the check once a minute and will trigger the do_something action if the folder is not empty of course:

* * * * *     if [ $(ls -1A isempty | wc -l) -gt 0 ] ; then do_something ; fi
share|improve this answer
add comment

Use inotifywait to wait for file changes:

inotifywait -m -e create /path
  while read file; do
    echo "$file"
    # do something here ./process.sh $file
  done

-m = monitor (don't exit after single event)
-e {event} = (only listen for create event), multiple (-e create -e modify)
path = (path to watch)

inotifywait documentation

share|improve this answer
1  
You're missing parts of that script—for example, the pipe. And you're not handling the output format of inotifywait correctly. –  derobert Mar 25 at 17:47
    
Did you try running it, i use this on ubuntu. –  dre Mar 26 at 18:22
    
You're missing a pipe (|) between /path and while, at minimum. –  derobert Mar 26 at 18:25
    
I use this script in production, it runs. What do you mean not handling the output format correctly, that is up to the script, you can handle it however you want. –  dre Mar 27 at 19:21
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.