1

I am trying to make a copy of completed files that are saved to a watch folder. I have created a bash script using inotifywait to watch the folder.

My original script worked great with files and folders/subfolders. Once I replaced the "-e create -e moved_to" with "-e close_write" it no longer recognizes/processes folders that are saved in the watch folder. It only works now with files. The reason I am needing to change to this is because a lot of my files are large (and multiples in folders) and takes some time to be saved completely (to the watch folder) and the script runs through all the processes before the files are completely saved/copied.

I have also tried the "wait" command but it does nothing at all. I know there are more complicated/better/correct ways of using it but I have not figured them out yet. After finding the "close_write" event in inotify I felt it would do exactly what I am needing by waiting to for the current process/command to finish before starting the next. And it works perfectly on large single files but is MIA (does nothing) whenever a folder is copied to the watch folder.

Here is a sample of my script. I am not well versed in writing bash scripts so sorry if it is a mess.

#!/bin/bash

WATCHED=/mnt/Watched
DESTINATION=/mnt/Destination
user=userid
group=groupid
perms=777

# Step 1 - Copying completed files to Destination folder

inotifywait -m -e close_write --format %f $WATCHED \
    | while read new
        do
            echo Detected new file $new, Copying to Destination folder
            cp -r "$WATCHED/$new" "$DESTINATION/"
            wait
                          
# Step 2 - Deleting unwanted files from Destination folder

            echo Deleting unwanted files from Destination folder
            find $DESTINATION -type f -iname "*.txt" -delete
            find $DESTINATION -type f -iname "*.nfo" -delete
            find $DESTINATION -type f -iname "*.website" -delete
            find $DESTINATION -type f -iname "*.exe" -delete
            find $DESTINATION -type f -iname "*.html" -delete
            find $DESTINATION -type f -iname "*.htm" -delete
            find $DESTINATION -type f -iname "*.sfv" -delete
            find $DESTINATION -type f -iname "*.parts" -delete
            find $DESTINATION -type f -iname "*.jpg" -delete
            find $DESTINATION -type f -iname "*.png" -delete
            find $DESTINATION -type f -iname "*.doc" -delete
            sleep 10
                           
# Step 3 - Change permissions of new files in Destination folder

            echo Changing ownership and permissions to Destination folder and files
            chown -R $user:$group "$DESTINATION"
            chmod -R $perms "$DESTINATION"
                           
        done

Any guidance would be appreciated.

Thanks

EDIT...

I cannot for the life of me get this script to even see a new folder added to the watch folder. It literally does NOTHING. If I replace the close_write with create & moved_to and make no other changes it sees the folder and contents and processes correctly. I find this very odd.

In fact I even tried to make a small test script to see if I can get it to work with a if/elif/else statement but once again when I copy a folder in the watch folder the script does nothing (doesn't even make to the loop). If I put a file in then it provides the correct output.

Here is the test script I ran. Can someone else confirm if they can get a new folder to be recognized and processed correctly?

#!/bin/bash
WATCHED=/mnt/Watched
inotifywait -re close_write --format '%w%f' -m $STEP1_WATCHED \
    | while read -r new
        do
            if [[ -d "$new" ]]; then            
            echo "Detected new folder $new"
            elif [[ -f "$new" ]]; then            
            echo "Detected new file $new"
            else
            echo "neither $new"
            fi
            done
    done
2
  • That's not hard to find out, by some testing, to see what events you need to monitor. Just print them while reproducing the cases. I guess you want close_write and moved_to ? – thanasisp Dec 15 '20 at 19:40
  • I have been testing this for days on end although I have not tried print yet as I am not sure of the correct syntax. I have tried to run both create as well as moved_to with close_write but then close_write does not work because the others are true. Meaning it will just start running the script without waiting for the files to finish copying first. Thanks for the help! – S. Ray Dec 15 '20 at 19:51
3

The reason you're not seeing directories created from your script is that you're looking for CLOSE_WRITE instead of CREATE with IS_DIR:

inotifywait .           # Terminal 1
Setting up watches.
Watches established.
./ CREATE,ISDIR zz

mkdir zz                # Terminal 2

Now, because CREATE can also apply to files you need to test the type of item as part of your decision making:

#!/bin/bash
watchDir=/mnt/watched

inotifywait -qre CLOSE_WRITE,CREATE --format $'%e\t%w%f' --monitor "$watchDir" |
    while IFS=$'\t' read -r events item
    do
        if [[ -d "$item" ]]
        then
            echo "New directory '$item'"

        elif [[ "$events" =~ CLOSE_WRITE ]]
        then
            if [[ -f "$item" ]]
            then
                echo "New file '$item'"

            elif [[ -e "$item" ]]
            then
                echo "New unknown item '$item'"
            fi
        fi
    done

This code will fail for file or directory names that contain newlines (touch $'break\nfile') or start with one or more tab characters (touch $'\ttabbed'), as inotifywait cannot use a format string that contains a null character (\000). There is also a small race condition where the event can fire but the target is potentially removed or even replaced before the loop can be processed and the target type evaluated.

3
  • roaima, Please see my reply to your comment above. I replied in the wrong section :-/ – S. Ray Dec 16 '20 at 1:27
  • 1
    @S.Ray I'm not entirely sure why you wanted to catch directory creation - if it's so that file copies have a structure to be written to, then a tool like rsync will create the intervening directories as part of a file copy and your original code will (mostly) work just as is. – roaima Dec 16 '20 at 9:58
  • roaima, I ended up changing my script completely and using rsync to copy my folders/files. Thanks! – S. Ray Dec 21 '20 at 3:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.