Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a system that is acting simultaneously as a test and demo platform. I'm recording a massive amount of data output by my systems that I wouldn't be recording when this is in deployment and because of that my log files are filling up very rapidly and I eventually run out of partition space on my harddrive.

I tried logrotate, but it seems to only work if my program isn't running. When the programs are down it properly truncates the file, records a compressed version, and discards the rest. However, when my programs that populate the log file are up and running it will properly create the compressed file, but it won't truncate the active file. I'm trying to figure out how to make it truncate.

I'm writing to the log files with a simple redirect. ProgramA>logA, programB>logB etc. My progam is outputting every time I receive an input vector, so it's outputting very rapidly. My assumption is that the logrotate is failing to truncate because of the constant writes to the field, but can anyone confirm rather that is the reason?

Where does logrotate keep it's error file anyways?

Also: I want to modify the behavior of logrotate. If it activates and sees a large, say 100 MB, log file I would like it to create a copy of only the last 1 MB of the file (discarding the older 99 MB of content of the file) before truncating the original log file. I need to keep recent data, but I don't care much about anything that is very old. Can anyone tell me how to do this?

share|improve this question
    
Have you considered cronolog? Pipe to that instead your current redirect, and it seems like that'd solve a lot of your problems. Also, if you really only care about the last 1MB, it sounds like you need a ring buffer logger of some sort. –  derobert Aug 28 '12 at 22:34
    
I'll look into cronolog right now, never heard of it before. Since the current 'logs' are all debug output that will be disabled in final release I was trying not to waste too much time writing a complicated system for handling the logging. If you know a linux program that will do the ring-buffering for me without my implementing I would definately look at it; but I don't want to write the logic for one. –  dsollen Aug 29 '12 at 0:24
    
You could write a close-enough to ring buffer program in a few lines of Perl. In fact, I think I shall do so as an answer... –  derobert Aug 29 '12 at 1:13
    
Also, I just found serverfault.com/questions/133320/… ... which has some more approaches. Including a kernel module (wtf). –  derobert Aug 29 '12 at 17:24
add comment

3 Answers 3

up vote 0 down vote accepted

As promised, a "close enough" to ring buffer program, in a few lines of Perl.

To use it, pass it the size of the log files you want, in bytes (give or take one line), and two or more log files. It'll switch between the log files every X bytes. I'm sure there is already a program out there to do this, but I couldn't find it with a quick search. E.g.,

$ your-app | log-splitter 1048576 /var/log/your-app/log1 /var/log/your-app/log2

assuming, of course, that the file log-splitter is executable, in $PATH, and contains:

#!/usr/bin/perl
use warnings qw(all);
use strict;
use autodie;
use Carp;
use Fcntl qw(SEEK_SET);
use IO::Handle;

# yes, we truncate all logs at start. Sorry.
@ARGV >= 2 or croak "Usage: $0 log-size log-file...";
my ($max_size, @lognames) = @ARGV;
my @logs = map {
    open my $fh, '+>', $_;
    $fh->autoflush(1);
    $fh;
} @lognames;

my $current_size = $max_size;
my $current_log  = $#logs;
while (defined(my $line = <STDIN>)) {
    if ($current_size >= $max_size) {
        $current_log  = ($current_log + 1) % @logs;
        $current_size = 0;
        seek($logs[$current_log], 0, SEEK_SET);
        truncate($logs[$current_log], 0);
    }

    $current_size += length($line);
    print {$logs[$current_log]} $line;
}

close($_) foreach @logs;

To clarify any licensing questions on that code:

CC0
To the extent possible under law, Anthony DeRobertis has waived all copyright and related or neighboring rights to the above program (log-splitter). This work is published from: United States.

share|improve this answer
    
This answered my full question, but I also wanted to thank angus for his simple and sweet solution for a lazy fix in the meantime. –  dsollen Aug 29 '12 at 16:53
    
Well, it's true that replacing > file with >> file is lazier than replacing it with | log-splitter 1234 file1 file2, but I would have counted that as an advantage :D –  angus Aug 29 '12 at 19:15
add comment

I know this is a little bit old but I had the same problem, and the solution was to use the append redirection to the output file rather than clobbering it. This way your application/script wont lock the file exclusively .

Instead of:

ProgramA > logA

Use:

ProgramA >> logA 

Regards.

share|improve this answer
add comment

From man logrotate:

   copytruncate
          Truncate the original log file to zero size in place after creat‐
          ing a copy, instead of moving the old  log  file  and  optionally
          creating  a  new one.  It can be used when some program cannot be
          told to  close  its  logfile  and  thus  might  continue  writing
          (appending) to the previous log file forever.  Note that there is
          a very small time slice between copying the file  and  truncating
          it,  so  some  logging  data  might be lost.  When this option is
          used, the create option will have no effect, as the old log  file
          stays in place.

That's probably the option you need to put in your configuration file.

About your other question: use the postrotate directive to run a command to truncate the file after it has been rotated. In that case, you should compress it manually at this stage, after truncating. (not really, this last part, because logrotate doesn't compress the first archive.)

EDIT: A little more about keeping the last 1 MB. You can configure it to rotate when the log file has more than, say, 200 K, and to only keep 5 backups. That way you'll always keep roughly the last 1 MB. If the first backup is too big, you can just wait until it "falls off" after 5 backups are done.

share|improve this answer
    
I suppose I should have clearified. I am using copytruncate command. However, it doesn't do the truncat as requested unless my programs are dead. –  dsollen Aug 29 '12 at 0:22
    
@dsollen Just redirect the output of your programs with >> instead of >. copytruncate will work then. –  angus Aug 29 '12 at 8:42
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.