extract fields from "tail -f" of a syslog stream

Question

Example line from syslog file:
Aug  1 10:25:50 10.10.10.1 id=firewall sn=XXXX time="2012-08-01 14:35:18 UTC" fw=x.x.x.x pri=6 c=1024 m=537 msg="Connection Closed" f=11 n=195273698 src=x.x.x.x:60246:X3: dst=x.x.x.x:443:X6:host.domain.com proto=tcp/https sent=181793 rcvd=649678

I want to do a

$ tail -f /var/log/log.log | SOMETHING

That will return

"src=x.x.x.x dst=x.x.x.x"

In plainspeak I want to be able to extract specific fields from text. I do not want to select these fields based on a field number (position in the line) because the order of things can change slightly with the system generating this file, so it doesn't work.

Open to suggestions but would prefer it to be a one-liner.

Answer 1

tail -f log | perl -e 'while (<>) { print "src=$1 dst=$2\n" if $_ =~ /src=([^\s]+) dst=([^\s]+)/; }'

Answer 2

sed 's/^.*\(src=[0-9.][0-9.]*\).*\(dst=[0-9.][0-9.]*\).*$/\1 \2/'

See Extracting a regex matched with 'sed' without printing the surrounding characters

Answer 3

grep has a --line-buffered option which allows you to follow tail -f

tail -f /var/log/messages | grep --line-buffered  'src=[0-9.]+\s+dst=[0-9.]+'

Answer 4

not quite as pretty but another solution implemented with awk:

tail -f /var/log/log.log | awk 'FS="src=" { print "src="$2 }' | awk 'FS=":" { print $1$4 }'`

Update: edited to be one line for tail -f

Answer 5

Or, improving a little on warlOck's:

tail -f log | perl -n -e 'print "$1 $2\n" if /(src=[^\s]+) (dst=[^\s]+)/;'

(I haven't figured out how to comment on an answer yet...)