Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I need to select certain lines from the log file and also save them to a text file. I tried the following but none of them are working as expected. The file "todel.txt" shows 0 bytes.

tail -f general.log | grep Some_word > >(tee -a todel.txt)

tail -f general.log | grep Some_word ; tee  todel.txt

tail -f general.log | grep Some_word | tee -a todel.txt
share|improve this question

migrated from serverfault.com Nov 18 '13 at 21:01

This question came from our site for professional system and network administrators.

    
Perhaps "Some_word" is case sensitive? In which case you should use grep -i. –  Sandeep Y Nov 18 '13 at 21:25
    
Are you sure, the required lines really appear there? By default tail -f will give you last 10 lines of input and whatever is written there unless the file is removed and recreated, in which case it quits. GNU tail has the --retry option which will wait until the file is recreated (although I think this option is non-POSIX). –  peterph Nov 25 '13 at 23:15

4 Answers 4

You need to add stdbuf(1) into your pipeline:

tail -f general.log | stdbuf -oL grep Some_word | tee -a todel.txt

This will set grep's stdout stream buffering mode to be line-buffered, otherwise grep waits to get at least 4096 bytes from the stream (this is the default on Linux for buffered i/o).

Alternatively, you can also call grep with --line-buffered:

tail -f general.log | grep --line-buffered Some_word | tee -a todel.txt

See http://www.pixelbeat.org/programming/stdio_buffering/ for an in-detail explanation.

share|improve this answer

The third one should be working fine assuming that you have access to write files in the current directory.

For example: tail -f general.log | grep "Some_word" | tee todel.txt

That is properly-defined tee syntax. If it's not working, you're doing something else wrong.

Also, it's a good idea to put your search string in quotes. If you have a filename matching your search string in the current working directory, it may be considered a file argument instead of a search string.

share|improve this answer
    
I think you mean tee instead of tail in your command line example. –  dmourati Nov 18 '13 at 6:23
    
Sure enough. Thanks. What I get for multi-tasking. –  Magellan Nov 18 '13 at 6:42
    
Using tail -f ... |grep allows one to grep the new lines incoming (-f option to tail). AFAIK grep has no such option. –  zhenech Nov 18 '13 at 7:34
    
You're right. Wasn't paying attention. –  Magellan Nov 18 '13 at 16:33
1  
Consider a text file with only a few characters in it. If you run tail -f... and pipe it into the grep | tee machinery, you would end up in exactly the scenario of the question: grep would wait for more input before processing it, while tee does not receive anything at all. Of course, if general.log is sufficiently large, the standard approach without altering the output buffer mode to line-buffered would work, but only if you it larger than the buffer size. And then you would need to wait for filling up grep's buffer again. See unix.stackexchange.com/a/102505/50954 –  tkrennwa Nov 25 '13 at 12:50
grep "my words" > mytextfile.txt
tail -f access.log > last_log.txt
tail -f access.log | grep "IP address" > hacker.log
share|improve this answer

It's possible the lines you're looking for are being sent to stderr instead of stdout, in which case ending the line with 2>&1 should fix the problem:

tail -f general.log | grep Some_word | tee -a todel.txt 2>&1
share|improve this answer
    
Why would tail send output to standard error? –  peterph Nov 25 '13 at 23:12
    
Oh, good point. I guess 2>&1 is just my reflexive answer to this type of question. –  Matthew Cline Nov 26 '13 at 0:38
    
You might want to update your answer then. Plus your 2>&1 actually refers to tee in this case. –  peterph Nov 28 '13 at 9:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.