Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a program whose output I redirect to a log file:

./my_app > log

I would like to clear (i.e. empty) the log from time to time (on demand) and tried various things like

cat "" > log

However it always seems that the original pipe is then disrupted and the program doesn't redirect its output to the log file anymore.

Is there some way to do that?

Update

Note that I cannot modify the application producing the output. It just spits it out to stdout and I want to save it in a log so that I can inspect it when I need it, and clear it when I want. However I shouldn't need to restart the application.

share|improve this question
    
that's why you usually use a logging daemon to log things... –  Kiwy Apr 3 at 10:54
    
@Kiwy can you elaborate on how that would solve the problem? –  inovaovao Apr 3 at 11:15
    
well you usually use a log daemon or let your app handle the log, because writing things to output and redirecting it is not reliable. you should take a look at syslogd or logrotate –  Kiwy Apr 3 at 11:21
2  
Do things work if you do ./my_app >> log (to force appending) and cp /dev/null log to truncate it? –  Mark Plotnick Apr 3 at 14:32
    
What error message do you get? What behavior do you see? "Doesn't redirect its output to the log file anymore" isn't very specific. Also, cat "" > log isn't a valid cat command since there's no file called "". –  Mikel Apr 3 at 15:52
show 1 more comment

4 Answers 4

Another form of this problem occurs with long running applications whose logs are periodically rotated. Even if you move the original log (e.g., mv log.txt log.1) and replace it immediately with a file of the same name before any actual logging occurs, if the process is holding the file open, it will either end up writing to log.1 (because that may still be the open inode) or to nothing.

A common way to deal with this (the system logger itself works this way) is to implement a signal handler in the process which will close and reopen its logs. Then, when ever you want to move or clear (by deleting) the log, send that signal to the process immediately afterward.

Here's a simple demonstration for bash -- forgive my cruddy shell skills (but if you are going to edit this for best practices, etc., please make sure you understand the functionality first and test your revision before you edit):

#!/bin/bash

trap sighandler INT

function sighandler () {
    touch log.txt
    exec &> log.txt
}

echo $BASHPID
exec &> log.txt

count=0;
while [ $count -lt 60 ]; do
    echo "$BASHPID Count is now $count"
    sleep 2
    ((count++))
done          

Start this by forking into the background:

> ./test.sh &
12356

Notice it reports its PID to the terminal and then begins logging to log.txt. You now have 2 minutes to play around. Wait a few seconds and try:

> mv log.txt log.1 && kill -s 2 12356

Just plain kill -2 12356 may work for you here too. Signal 2 is SIGINT (it's also what Ctrl-C does, so you could try this in the foreground and move or remove the logfile from another terminal), which the trap should trap. To check;

> cat log.1
12356 Count is now 0
12356 Count is now 1
12356 Count is now 2
12356 Count is now 3
12356 Count is now 4
12356 Count is now 5
12356 Count is now 6
12356 Count is now 7
12356 Count is now 8
12356 Count is now 9
12356 Count is now 10
12356 Count is now 11
12356 Count is now 12
12356 Count is now 13
12356 Count is now 14

Now let's see if it is still writing to a log.txt even though we moved it:

> cat log.txt
12356 Count is now 15
12356 Count is now 16
12356 Count is now 17
12356 Count is now 18
12356 Count is now 19
12356 Count is now 20
12356 Count is now 21

Notice it kept going right where it left off. If you don't want to keep the record simply clear the log by deleting it

> rm -f log.txt && kill -s 2 12356

Check:

> cat log.txt
12356 Count is now 29
12356 Count is now 30
12356 Count is now 31
12356 Count is now 32
12356 Count is now 33
12356 Count is now 34
12356 Count is now 35
12356 Count is now 36

Still going.

You can't do this in a shell script for an executed subprocess, unfortunately, because if it is in the foreground, bash's own signal handlers (traps) are suspended, and if you fork it into the background, you can't reassign its output. I.e., this is something you have to implement in your application.

However...

If you can't modify the application (e.g., because you did not write it), I have a CLI utility you can use as an intermediary. You could also implement a simple version of this in a script which serves as a pipe to the log:

#!/bin/bash

trap sighandler INT

function sighandler () {
    touch log.txt
    exec 1> log.txt
}

echo "$0 $BASHPID"
exec 1> log.txt

count=0;
while read; do
    echo $REPLY
done  

Let's call this pipetrap.sh. Now we need a separate program to test with, mimicking the application you want to log:

#!/bin/bash

count=0
while [ $count -lt 60 ]; do
    echo "$BASHPID Count is now $count"
    sleep 2
    ((count++))
done           

That will be test.sh:

> (./test.sh | ./pipetrap.sh) &
./pipetrap.sh 15859

These are two separate processes with separate PIDs. To clear test.sh's output, which is being funnelled through pipetrap.sh:

> rm -f log.txt && kill -s 2 15859

Check:

>cat log.txt
15858 Count is now 6
15858 Count is now 7
15858 Count is now 8

15858, test.sh, is still running and its output is being logged. In this case, no modifications to the application are needed.

share|improve this answer
    
Thanks for the nice explanations. However in my case I cannot modify the application to implement your solution. –  inovaovao Apr 3 at 12:38
1  
If you can't implement a signal handler in your application (because you can't modify it period), you can use this technique to pipe the log through a signal trap -- see the stuff after "However..." –  goldilocks Apr 3 at 12:56
    
Ok I'll give it a try and let you know how it went. –  inovaovao Apr 3 at 14:11
    
I finally have a CLI app written in C for this (sorry it took a bit longer than originally intended): cognitivedissonance.ca/cogware/pipelog –  goldilocks Apr 28 at 13:44
add comment

As fast solution you can use a log with rotation (daily rotation for example):

date=date +%Y%m%d LOGFILE=/home/log$date.log

and redirect logging to it ./my_app >> log$date.log

share|improve this answer
    
I'd like to be able to rotate on demand. This is actually a log that is produced during an automated test and I would like to clear it before running the test. –  inovaovao Apr 3 at 12:30
add comment

If I'm understanding correctly, tee seems like a reasonable approach:

$ ./myapp-that-echoes-the-date-every-second | tee log > /dev/null &
[1] 20519
$ head log
Thu Apr  3 11:29:34 EDT 2014
Thu Apr  3 11:29:35 EDT 2014
Thu Apr  3 11:29:36 EDT 2014
$ > log
$ head log
Thu Apr  3 11:29:40 EDT 2014
Thu Apr  3 11:29:41 EDT 2014
Thu Apr  3 11:29:42 EDT 2014
share|improve this answer
add comment

This is a problem that has long been solved with syslog (in all it's variants) but there are two tools that would solve your particular problem with a minimum of effort.

The first, more portable but less versatile solution is logger (a must have for any administrators toolbox). It is a simple utility that copies standard input to syslog. (passing the buck, and making file rotation the problem of logrotate and syslog)

The second, more elegant but less portable solution is syslog-ng which in addition to accepting log messages from the standard syslog sockets can execute programs which output is filtered through the logger. (I have not used this feature yet, but it looks perfect for what you are wanting to do.)

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.