Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am working on some batch scripts involving the following:

  • Run some non-terminating sub-processes (asynchronously)
  • Wait for t seconds
  • Perform other task X for some time
  • Terminate subprocesses

Ideally, I would like to be able to differentiate the stdout of the sub-processes which has been emitted before X from that which has been emitted after X.

A few ideas come to mind, although I have no idea as to how I would implement them:

  • Discard stdout for t seconds
  • Insert some text (for instance, 'Task X started') to visually separate the sections
  • Split stdout into various output streams
share|improve this question
1  
Is it a requirement for both X and asynchronous sub-process to write to the same file? –  jw013 Jul 12 '12 at 20:04
    
No not necessarily. –  WaelJ Jul 12 '12 at 22:38
add comment

1 Answer 1

up vote 2 down vote accepted

While you could complicate the matter with exec and extra file descriptor wrangling, your second suggestion is the simplest. Before starting X, echo a marker string into the log file.

All those commands would be appending to the same file, so maybe it would be a good idea to prepend all output of X with a marker, so you can tell apart its output from the one of the still running previous commands. Something along the lines of:

{ X; } | sed 's,^,[X say] ,'

This would make further analysis much simpler. It is not safe and for very verbose programs, race conditions would happen often.

If you're willing to take the chance to break one log line and can interrupt the first batch of apps without consequence, this would work too:

{ Y; } >> log &
sleep $t
kill -STOP %% # last job, like the same as %1
echo -e "\nX started" >> log
kill -CONT %%
{ X; } >> log2
share|improve this answer
2  
Sounds like a reasonable suggestion, but is it not possible that different outputs might get mangled (within a line). Something like: X say output from x OUTPUT FROM OTHER PROCESS Comes out as: x sa**_OUTPUT FROM OTHER PROCESS_**y output from x –  WaelJ Jul 12 '12 at 14:16
    
You are right. I tried "yes" and "yes no" into the same file, which resulted in about 300M of lines by the time I killed them. uniq-ing that gave me "y", "no", "o", "ny", "noy". The number of bad occurences were very close, but not equal, so even an EOL marker would not help if you have apps logging as ultrafast as yes. –  lynxlynxlynx Jul 12 '12 at 16:28
    
I like your idea with interrupting the process, but I am not sure whether that is acceptable in my case. The background processes are making network requests and receiving data. My first concern: will the network interaction be affected? Second: this would probably disrupt benchmarking the bandwidth usage (which is what X does, actually). I should be able to live with your original suggestion. I did a quick test and line mangling did not appear to be an issue in my case. Nonetheless, I will wait for a bit and see if anyone can find a more general and robust solution :) –  WaelJ Jul 12 '12 at 23:08
    
I'm afraid that even swapping file descriptors would incur a few microseconds of interruption. In both cases, unless you're working with really short time spans, I can't see how this would create a significant measurement error or bust any internal timeouts. –  lynxlynxlynx Jul 13 '12 at 6:50
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.