Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

If I have a process writing to nohup and the storage where it writes to is slow, does it block the process which is running in nohup? For example:

process A is running in nohup, it needs to write 10 lines to console

it takes 2 minutes for disk storage to write these 10 lines into nohup.out (it write 6 lines and then it stuck for a minute waiting for disk)

is the process A blocked until the remaining lines can be written, or not?

share|improve this question

2 Answers 2

up vote 1 down vote accepted

nohup mode has no relevance on whether the process is I/O bound. So yes, slow storage will probably slow down the process.

If the process has bursty output (e.g. 10 lines every 1 hour), but throughput lower than your "slow" storage, then you can improve performance by e.g. writing to a pipe or a FIFO (which then gets copied to the slow storage by another process), as long as one burst of output doesn't exceed the pipe's buffer.

mkfifo /tmp/fifo
nohup prog >/tmp/fifo &
cat </tmp/fifo >slow_log_file &

or

nohup prog | buffer -s 16k >slow_log_file &

where buffer is a special program (available on Debian with apt-get install buffer) that maintains a large internal queue to avoid blocking the writer.

Or if you control the source code, you could implement buffering yourself and maybe use a separate logger process to write to the slow storage. I know djbdns (famous for its stability) uses a logger process.

share|improve this answer
    
is there some utility which can do this? something like tee, but it would buffer the input into memory and write to a file using separate thread or such –  Petr Jan 14 '13 at 13:19
    
I can change the sources but I don't like idea that application itself would handle this, because in case of crash (or kill) the application wouldn't finish writing to log and there could be some data missig –  Petr Jan 14 '13 at 13:21
    
OK, suggested a logger process instead of a thread. Or just go with mkfifo or with buffer –  dan3 Jan 14 '13 at 14:26

As a normal process which produce huge IO it will be blocked. But usually filesystem blocks are cached in memory. And when OS try to write dirty blocks to disk you will see that wait column in top output become 100%

share|improve this answer
    
I think OS caching depends on the filesystem type, though. With such throughput this is probably not regular storage, but something like nfs (or could be as exotic as sshfs) –  dan3 Jan 14 '13 at 12:01
    
I wrote "usually". :) Of course we can use directio or O_DIRECT in open() for disable caching –  dchirikov Jan 14 '13 at 12:40
    
Right :) but we are not in a "usually" scenario (10 lines in 2 mins). I suspect a network filesystem, but the OP isn't saying. I know there are solutions for NFS caching in that case –  dan3 Jan 14 '13 at 14:29

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.