Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've got a legacy app that is generating a huge lot of logs message. I'd like to be able to rotate the file every minute (or so) but without loosing / truncating any message. The app is not able to change of file by itself so I need to do it externally. Surfing the net I've seen a lot of suggestion to use cp followed by a cat /dev/null but since my app is generating almost a continuous flow of messages some messages are lost during the process.

Do you have any ideas ?

share|improve this question
You can also rotate the log file when file size reaches a specific limit – Raza May 28 '13 at 21:04
A side comment: cat /dev/null is a popular still urban legend. It can be easily replaced by : or even nothing, given the fact it is a no-op. – jlliagre May 28 '13 at 21:47
size doesn't matter in my case : I need it to rotate as frequently as possible to save auditing logs remotely. – Cerber May 29 '13 at 16:48
@jlliagre "easily replaced by" meaning: ": is faster by a factor of 200" (on my system) – Hauke Laging May 29 '13 at 17:50
why not just logrotate -f config ? – nurettin Mar 19 '15 at 14:52
up vote 5 down vote accepted

Make the log file a FIFO (man mkfifo) and put a process on the reading side which separates its input into files of limited size.

mkfifo /path/to/logfifo.app_xy
split ... </path/to/logfifo.app_xy &
share|improve this answer
Nice start but I'm looking for a complete solution. Not that yours is bad, but I was very surpised that in 2013 the minimum rotation time for every one is daily like in logrotate. Everybody is talking about big data, auditing and so on, but I feel like the only one needing much frequent rotation ... – Cerber May 29 '13 at 16:54
@Cerber "complete solution" meaning what? logrotate can rotate by size, too, if I remember correctly. – Hauke Laging May 29 '13 at 17:32
ooooooops, you meant THE split unix core command. Sorry @HaukeLaging I've read your answer too fast. I'll check that on my app – Cerber May 29 '13 at 17:46
Hi Hauke Laging, I went on with a couple of tests but I can't get it right : split is not fit for time-based split, so I've thought to 'crontab' it. But I started with a simpler solution : I wrote a perl script that outputs in the fifo each numbers from 0 to 10000 with a 100ms sleep. I ran this in background and did a head on it to gather the first bytes just to check that two head calls would not break the sequence ... unfortunalty I got a [1]+ Broken pipe perl script.pl >log at the first head. Therefore I think split would not work. What am I missing ? – Cerber Jun 4 '13 at 18:14
@Cerber You probably did something like head log; head log. Indeed, the first head closes the FIFO then. You need a wrapper around the processes which read from the FIFO: {head log; head log;} or exec 3<log; head <&3; head <&3 – Hauke Laging Jun 4 '13 at 20:07

Maybe the logrotate tool can handle this.

share|improve this answer
logrotate needs at least a little support from the app, due to the way files work in Unix. If the log file is renamed without notifying the app in some way, the app will continue to write to the renamed file, rather than starting to write to a new file with the old name. man 8 logrotate and note the "postrotate" command in the examples. – Jander May 29 '13 at 8:43
beside, logrotate minimum is daily ... which is very far from what I want to achieve – Cerber May 29 '13 at 16:50
Actually you can force rotation depending on the file size. But it is true that you need either the application's help or the restart it for the log to be rotated. – Spack May 29 '13 at 17:50

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.