How can I limit the size of a log file written with >> to 200MB?

$ run_program >> myprogram.log
  • Do you want the program killed after 200MB? Or do you want the last 200MB with everything older in the bit bucket? – Aaron D. Marasco Jul 23 '11 at 13:10
  • nope, the process can't stop until manually killed – david Jul 25 '11 at 2:35
  • decided to go for logrotate, thanks everyone for the valuable inputs – david Jul 25 '11 at 3:18

If your application (ie. run_program) does not support limiting the size of the log file, then you can check the file size periodically in a loop with an external application or script.

You can also use logrotate(8) to rotate your logs, it has size parameter which you can use for your purpose:

With this, the log file is rotated when the specified size is reached. Size may be specified in bytes (default), kilobytes (sizek), or megabytes (sizem).

  • 1
    +1 Logfiles with repeated information can often get compressed in orders of magnitude. – user unknown Jul 22 '11 at 15:28
  • Does logrotate truncate files, or simply copy or move them? Because this will only work if the file associated with the fd is truncated. If it is mv'd, the file will continue to grow, if it is unlink'd, it will just be held open and continue to grow until the process exits... IIRC logrotate copies, unlinks, and creates a new file, which is why one often has to send SIGHUP to dæmons when their logs have been rotated. – Michael Trausch Jul 22 '11 at 16:26
  • Note that even if it's truncated, there's a problem if the file was not opened in append mode (log files should always be opened in append mode, but in my experience are often not) – Random832 Jul 22 '11 at 20:17
  • Logrotate can rotate once the file has reached a certain size, daily, weekly, etc. It can also compress, and you can use the postscript option to have the logrotate config send then SIGHUP to the program. – laebshade Jul 23 '11 at 21:48

If your program doesn't need to write any OTHER files that would be larger than this limit, you can inform the kernel of this limit using ulimit. Before you run your command, run this to setup a 200MB file size limit for all process run in your current shell session:

ulimit -f $((200*1024))

This will protect your system but it might be jaring for the program writing the file. As eyazici suggests, consider setting up logrotate to prune log files once they reach a certain size or age. You can discard old data or archive it for a period of time in a series of compressed files.

  • this limits the size of any file written by the program – Kim Jul 22 '11 at 10:45
  • True. If the program had a legitimate need to write other large files, you would need a different solution. – Caleb Jul 22 '11 at 10:46

You may create a new filesystem image, mount it using loop device and put the log file on that filesystem:

dd if=/dev/zero of=./200mb.img bs=1024 count=200000 # create new empty 200MB file
mkfs.ext2 200mb.img # or ext3, or whatever fits your needs
mkdir logs
sudo mount -t ext2 -o loop 200mb.img logs # only root can do '-o loop' by default
run_program >>logs/myprogram.log

You may also use tmpfs instead of a file, if you have enough memory.

  • Creative idea... that leaves it up to the program as to what to do when it runs out. – Aaron D. Marasco Jul 23 '11 at 13:05

You can truncate the output with head:

size=$((200*1024*1024-$(stat -c %s myprogram.log)))
run_program | head -c ${size} >> myprogram.log
  • Very creative. Just a note that this would only work to limit NEW data being written to the file, it would not take into account how large or small the file already was. – Caleb Jul 22 '11 at 11:15
  • 2
    Note that it is likely that this will kill the program (with SIGPIPE) once it has reached the size limit, rather than discarding the data. – Random832 Jul 22 '11 at 11:38
  • 1
    I was thinking similar with some dd magic but yeah @Random832 is right, you'll get a SIGPIPE as head/dd/whatever drops it. – Aaron D. Marasco Jul 23 '11 at 13:07
  • What about ignoring it with a trap '' SIGPIPE ? – tuxce Jul 25 '11 at 15:35
  • Or pipe instead to { head -c "$size" >> log; cat > /dev/null; }. – Stéphane Chazelas Jan 22 '13 at 11:10

In package apache2-utils is present utility called rotatelogs, it may be helpful for you.


rotatelogs [ -l ] [ -L linkname ] [ -p program ] [ -f ] [ -t ] [ -v ] [ -e ] [ -c ] [ -n number-of-files ] logfile rotationtime|filesize(B|K|M|G) [ offset ]


your_program | rotatelogs -n 5 /var/log/logfile 1M

Full manual you may read on this link.


Since it is text, I would write a script in your favorite language and pipe it to that. Have it handle the file I/O (or keep it all in memory and then dump it on SIGHUP or similar). For that, instead of 200MB I would think of a 'reasonable' number of lines to keep track of.

  • Keeping 200MB of log data in memory for no other reason than to be able to truncate it isn't a very good use of system resources. Nor is doing a line count on a big log file. I would recommend using tools built for this like syslog and logrotate. – Caleb Jul 25 '11 at 8:45

The following script should do the job.

while getopts "s:n:" opt; do
  case "$opt" in
shift $((OPTIND-1))
if [ $# == 0 -o -z "$1" ]; then
    echo "missing output file argument"
    exit 1
while :; do
    dd bs=10 count=$(($LOG_SIZE/10)) >> $OUT_FILE 2>/dev/null
    SZ=`stat -c%s $OUT_FILE`
    if [ $SZ -eq 0 ]; then
        rm $OUT_FILE
    echo -e "\nLog portion finished" >> $OUT_FILE
    NUM=$(($NUM + 1))
    [ $NUM -gt $NUM_SEGM ] && NUM=1

It has a couple of obvious short-cuts, but overall it does what you asked for. It will split the log into a chunks of a limited size, and the amount of chunks is limited too. All can be specified via the command-line arguments. Log file is also specified via the command line.

Note a small gotcha if you use it with the daemon that forks into background. Using a pipe will prevent the daemon from going to background. In this case there is a (likely bash-specific) syntax to avoid the problem:

my_daemon | ( logger.sh /var/log/my_log.log <&0 & )

Note the <&0, while seemingly redundant, it won't work without this.


I'm certain the original poster has found a solution. Here's another one for others that may read this thread...

Curtail limits the size of a program's output and preserves the last 200MB of output with the following command:

$ run_program | curtail -s 200M myprogram.log


NOTE: I'm the maintainer of the above repo. Just sharing the solution...

  • I love the idea of curtail. I'm not as familiar with C, so it's there a chance to provide a binary for it? or at least detailed instructions on how to install it? – Felipe Jan 10 at 0:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.