Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have an OSX GUI application which I can't change - it writes new CVS lines to a log file when it completes a job. I'd like to work with these new lines, but they are constantly being added.

Is there a way to redirect the application's output directly into a script where I can then process the new lines as it generates them? Reading the file every X seconds to pick up changes isn't desirable, and while I could use launchd to detect changes to the file and process them that way - I'm looking for something a bit more elegant. Any ideas (with examples - I'm learning) would be much appreciated.

EDIT

I'm not sure it's the best approach, but I've managed to get instant results via tail's -F flag. While I'm able to filter for the last line, the first time it runs, when the file is updated, it outputs all the lines not just the last one (as desired).

tail -F -n 2 '/path/to/Epson4880_audit.txt' | grep "."
share|improve this question
    
So it looks like there are two different approaches - some sort of file watching utility: launchd, fswatch (some of which are new to me) or a FIFO pipe. Having looked at FIFO im not sure if what Im doing actually requires the overhead and complexity (I might just be skittish here). That said with a file watch system, I have to develop a mechanism to prevent duplicate entries, and race conditions if new lines written are written in rapid succession. –  orionrush Jun 21 '13 at 22:27
    
You could use read to process each new line (use tail with -n0 to ignore the existing content): tail -F -n0 Epson4880_audit.txt | while read line; do "process line here"; done –  don_crissti Jun 21 '13 at 22:31
    
I have the same problem with this as I did with my own implementation in the post. Basically tail doesn't seem honor the -n0 as I was expecting - every time the file saves, the entire file is read rather then just the last line. –  orionrush Jun 21 '13 at 23:31
    
@don_crissti at least with a single invocation of tail, there dosen't seem to be a way to just get just the last line when using ` tail -F -n0`. It will deliver the last line, the fist time, but on every file update it outputs the whole file. Happy to be proved wrong. See: stackoverflow.com/questions/17246146/… –  orionrush Jun 22 '13 at 13:39

2 Answers 2

up vote 4 down vote accepted

You could replace the output file by a FIFO and make your other program get its input from it.

For example:

mkfifo Epson4880_audit.txt
# launch your program
grep "." Epson4880_audit.txt

Note this is not guaranteed to work. Some program check that their output is a real file. And if you do want to keep a log of the lines, the second process has to make sure to print them as they arrive. For example doing tee real_Epson4880_audit.txt | grep ".". Please also note that you won't be able to launch your first program without the second because the first process will be forced to wait for a reader. In the case you want to run the first program without the second, you'll have to remove the FIFO first.

share|improve this answer
    
Right, I had in mind the case of a RDWR FIFO. I correct the answer. –  lgeorget Jun 21 '13 at 21:13
    
Im going to mark this the correct answer as you precisely address my question about redirecting file writes. Ultimately I'll probably use @slm's approach and monitor the file with the likes of fswatch because it seems simpler, and less work (in some respects) to implement. –  orionrush Jun 22 '13 at 13:52

Most operating systems have a facility where you can subscribe to the activity on a file. In Linux this facility is built into the kernel and is called iNotify. You could create something along those lines to watch the file for activity and then handle these updates.

Here are some resources to get you started with understanding iNotify:

Even though these pages show C/C++ code you do not have to develop your own tools to interact with it. There are command line tools that you can use to subscribe a file and/or directory and then receive notifications.

NOTE: On OSX you'll use something similar called kqueue. I found this thread that discusses a lot of the options over on Stack Overflow, titled: Is there a command like “watch” or “inotifywait” on the Mac?.

There are tools that offer similar facilities to inotifywait on OSX such as launchd, kqwait, fswatch or watchdog.

The other approach that you could take with this is to look at how monitoring tools such as Nagios monitor log files for activity. There is a wealth of plugins to Nagios that do exactly what you're trying to do. Some of them monitor a file just for activity while others monitor the files for specific strings. Take a look at the plugins page.

Nagios Plugins - Category: Log Files

share|improve this answer
    
iNotify is Linux only, the OP mentioned OSX, where I suppose it's kevent/kqueue like on BSDs. –  Stéphane Chazelas Jun 21 '13 at 21:14
    
the 'i' in iNotify threw me, sadly its an OSX environment - thanks for the tip though - it might help someone else. –  orionrush Jun 21 '13 at 21:18
    
@orionrush - see the link I provided on SO. There are other tools such as fswatch and watchdog. –  slm Jun 21 '13 at 21:22
    
some good tool tips there, I'll have to do some research... –  orionrush Jun 21 '13 at 22:13
    
Unfortunately, I don't have the points to up-vote this. Its a solid suggestion and you've sketched out the landscape very well. Thanks for adding in the thread on OSX equivalents. –  orionrush Jun 22 '13 at 13:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.