3

I have a folder A in which every now and then new files appear. It may happen that there are no new files for an hour, but also that there are several new files each second. All files have a unique timestamp in their name and different file extensions. What, how and when something is put in the folder cannot be modified. It's an directory on my pc which is accessed from the outside via ftp.

In another folder B I would like to have a copy of the most recent jpg file from folder A called "Newest.jpg" as soon as there is a new one. 1 second delay would be fine. It is supposed to imitate the output of a webcam.

What is the best and least computational expensive way to achieve this on a Raspberry-like system running Ubuntu 14.04 LTS?

3
  • The easiest way would be to edit the code or script that is putting new jpegs in the directory, and add a line: rm latest; ln -s "file-just-added-timestamp" latest.
    – Wildcard
    Commented Oct 11, 2016 at 9:41
  • @Wildcard Yes, of course. But what, how and when something is put in the folder is not modifyable for me. It's an ftp folder.
    – Rob
    Commented Oct 11, 2016 at 9:45
  • @MarkPlotnick It's the same ext4 system which contains the Ubuntu System. It's on the same pc. My pc runs the ftp server.
    – Rob
    Commented Oct 11, 2016 at 10:23

2 Answers 2

3

If you install the package inotify-tools you can write a simple script that will get inotify events when a new file is created.

inotifywait  -m -e create --format %f A | 
while read filename
do    ln -f A/"$filename" B/Newest.jpg
done

Whenever a new file is created in the watched directory A, the event is printed in the given format %f i.e. just the filename. The shell loop reads this and creates a link. There should be no delay between the file creation and the link being made.


If you need to exclude certain filenames from matching you can add --exclude and a regex pattern to the inotifywait, eg --exclude '.*.wav$' to exclude *.wav files. Alternatively, it might to simpler to test for a wanted *.jpg pattern inside the shell loop. If you are using bash you could simply add at the start, after the do:

[[ "$filename" != *.jpg ]] && continue

This tests against a glob pattern rather than a regex.

3
  • Thanks, I actually expected something like this in the beginning. It seems more flexible to me. Should I add sleep 1 if I put this in a loop? I guess not, or do I? Do I need an outer loop at all? Without an outer loop, will your script ever finish? And how to restict the notification just to new jpg files?
    – Rob
    Commented Oct 11, 2016 at 15:14
  • No sleep nor outer loop is needed, the inotifywait runs forever. It only prints the names of files created after it has started; existing files are of no consequence. I updated my answer on how to ignore the creation of new files in the A directory that are not *.jpg files, and are to be ignored.
    – meuh
    Commented Oct 11, 2016 at 15:46
  • 2
    Rather than rm -f you might use ln -f instead. I don't know if that makes it fully atomic, but I expect it will result in smaller windows during which Newest is unavailable.
    – Wildcard
    Commented Oct 11, 2016 at 15:59
3

First, make a symlink to any file in the directory, and call it "latest":

ln -s somefile.jpg latest

Then run the following command in the directory as often as you like:

find . -type f -newer latest -exec ln -fs {} latest \;

This is actually rather elegant, because the evaluation sequence doesn't matter. At the end of this command latest will be a symlink to the newest file in the directory. EDIT: This is wrong. According to POSIX:

The -user, -group, and -newer primaries each shall evaluate their respective arguments only once.

This rather invalidates this approach, but you could still make something out of it. However, given this fact, I would use inotify as suggested in the other answer.


I don't know how many CPU cycles it would burn, but just running:

while true; do find . -type f -name '*.jpg' -newer latest -exec ln -fs {} latest \; ; done

will make sure latest always points to the most recent jpeg file—as fast as possible. If you don't expect new images more often than once per second, you could use while sleep 1 instead of while true.


Note that as the number of files in your directory grows, this will get more and more inefficient. I have a few ideas for how you could handle that, but since it's on ftp they may not be applicable.

My first thought was to move the older files into another directory as part of the same find command, so they won't have to be evaluated again. The problem with that is, the file that the latest symlink points to counts as "older," since it's not newer. We can't move that file or we'll break the symlink!

However, since we are only evaluating the files named .jpg, we can get around this by making latest a hard link rather than a symlink:

mkdir ../oldfiles
while sleep 1; do find . -type f -name '*.jpg' \( -newer latest -exec ln -f {} latest \; -o -exec mv {} ../oldfiles \; \)

Again, I don't know how this will go on ftp, but for local files it should work great.

5
  • Adding a sleep 1 before done would help on the CPU.
    – Tigger
    Commented Oct 11, 2016 at 9:56
  • I don't see how you ensure that only new jpg files are taken into account and files with other extensions are ignored. Am I missing something?
    – Rob
    Commented Oct 11, 2016 at 10:00
  • @Rob, sorry. In my own directory organization I would never put other files into a dedicated directory that is supposed to only hold jpegs...but I've made that explicit now.
    – Wildcard
    Commented Oct 11, 2016 at 10:09
  • @Tigger, see edit. You can do that in the while test more concisely.
    – Wildcard
    Commented Oct 11, 2016 at 10:10
  • Thanks! The ftp server is the pc on which the script is supposed to run. So no problem here. Runs perfectly!
    – Rob
    Commented Oct 11, 2016 at 11:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .