60 reputation
27
bio website
location
age
visits member for 2 years, 9 months
seen Mar 7 '12 at 23:28

Aug
5
awarded  Notable Question
Jun
2
awarded  Notable Question
Mar
22
awarded  Popular Question
Mar
6
awarded  Popular Question
Mar
7
awarded  Commentator
Mar
7
accepted process files in a directory as they appear
Mar
7
comment process files in a directory as they appear
Using inotify wait, and running a string of commands (as described here gnu.org/software/parallel/man.html#example__composed_commands ) was my final solution
Mar
7
comment process files in a directory as they appear
After exploring what alerts get raised on it, i would agree, that it is just an open but empty file as data is being streamed into it. using CLOSE_WRITE was perfect for my situation.
Mar
6
comment process files in a directory as they appear
Answering my own comment, you can escape characters with a quoted (aka, out of parallel execution ). so my parallel looks like parallel ' awk '\''{ print $1 } '\''...
Mar
5
comment process files in a directory as they appear
And my next comment would how can I use something like awk (needing to be escaped by single quotes) in a parallel statement where I want to send multiple commands (therefore needing to enclose the commands in single quotes as well, such as your top example. ........... Simple use case is: ls ~ | parallel ' du -k | awk '{ print $1 }';ls -l'
Mar
5
comment process files in a directory as they appear
Ole, the touch actually comes from parallel it seems. I have parallel which operates on a list of files, changing their format and contents. However, it quickly runs through the inputs creating output files, and then slowly processes the inputs. (You can see my earlier process here: unix.stackexchange.com/questions/32162/… )
Mar
1
comment process files in a directory as they appear
How will the inotify interface handle my issue of touching files (aka, creating zero byte versions), and me only wanting to be notified (or take action on) the complete files)
Feb
29
asked process files in a directory as they appear
Feb
29
awarded  Editor
Feb
29
revised using parallel to process unique input files to unique output files
added 1113 characters in body
Feb
28
comment using parallel to process unique input files to unique output files
Great answer (and major points for reading my request of using parallel).
Feb
28
awarded  Scholar
Feb
28
accepted using parallel to process unique input files to unique output files
Feb
23
awarded  Supporter
Feb
21
comment using parallel to process unique input files to unique output files
How is the 'wait' smart enough to take the same file that is currently being iterated over and re-enter the sibling "if" statement?