13

I want to get just e-mail addresses that end in "@xyz.nl" from my mail logfile. To achieve this I do:

# tail -f /var/log/mail.log | grep --i --line-buffered "@xyz.nl" | cut -d '@' -f 1 | cut -d '<' -f 2

The --line-buffered with grep is necessary because it will otherwise buffer its output because the pipe is not considered a terminal. Grep will output lines like these:

Aug 29 11:56:01 localhost postfix/smtp[4124]: 05491500123: to=<[email protected]>, relay=123.456.123.456[123.456.123.456]:25, delay=2, delays=0.4/0/0.4/1.2, dsn=2.0.0, status=sent (250 2.0.0 u7T9twxN074009 Message accepted for delivery)

The first cut then makes:

Aug 29 11:56:01 localhost postfix/smtp[4124]: 05491500123: to=<someone

The second cut should generate:

someone

However it seems that cut is also buffering. If I start the command with cat instead of tail -f I get all relevant results (in the prefered format) from the log file. But I need the results from the log file in real time.

I tried using unbuffer for this:

# tail -f /var/log/mail.log | grep --i --line-buffered "@xyz.nl" | unbuffer cut -d '@' -f 1 | cut -d '<' -f 2

Also tried:

# unbuffer tail -f /var/log/mail.log | grep --i --line-buffered "@xyz.nl" | unbuffer cut -d '@' -f 1 | cut -d '<' -f 2

...which should remove the 4K buffering from the first cut. However, this doesn't work. I know it is buffering because if I grep for our local domain it gets a lot more hits, the buffer is filled sooner and output is generated earlier (in 4K batches).

So my question is: how do I unbuffer cut?

Related: I know sed and (g)awk can deliver the e-mail addresses to me. I have been trying but as yet without any result. Answers using sed or (g)awk are welcome and may solve my direct issue but I remain interested in the nominal answer of the question how to unbuffer the cut command. The cut command doesn't speak of (un)buffering.

4
  • awk -F'[><@]' '/@xyz.nl/ {print $2}'...
    – jasonwryan
    Sep 2, 2016 at 7:02
  • 1
    try grep -oP '[^<]+([email protected])' (along with your other grep options as needed)
    – Sundeep
    Sep 2, 2016 at 7:08
  • Both the awk and grep options above work like a charm!
    – Forkbeard
    Sep 2, 2016 at 7:31
  • Still unanswered: Why does’t unbuffer work here? Dec 12, 2020 at 16:42

1 Answer 1

28

If you're on a system using GNU Coreutils (almost any Linux), you can try stdbuf:

… | stdbuf -oL cut -d '@' -f 1 | …

-oL makes it line buffered, which seems like what you want.

3
  • This is exactly what I was looking for, thanks!
    – Forkbeard
    Sep 2, 2016 at 7:35
  • Does not change anything for me for some reason. :(
    – panzi
    Jun 29, 2017 at 10:32
  • 1
    @panzi Feel free to ask your own question, linking to this one. Please include full details (exact command you're running if possible, OS, distro, and version, etc.) Hopefully someone will be able to figure out why it doesn't work for you.
    – derobert
    Jun 29, 2017 at 16:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .