0

Im looking to parse a log using a perl command that I designed. So far all I would like to accomplish is pulling a specific timestamp format and retrieve the host. the log would look like this

2016-05-07T09:07:04.933343+00:00 heroku[router]: status=301 bytes=680 service=2698ms connect=1ms dyno=web.2 fwd="10.29.10.29" at=info host="jamaican.com" request_id=32fc8d88-99f8-4cc2-89f9-284d059eebf8 method=GET path="/blog"

my command so can parse the date format I want but I cant seem to figure out how to add the host in the output. any recommendations would be great!

cat test.log  |
perl -lne 'print $1 if  /^([0-9]+[-]+[0-9]+[0-9]+[-]+[0-9]+[0-9]+[T]+[0-9]+[0-9]+[:]+[0-9]+[0-9]+[:]+[0-9]+[0-9])/'
2
  • i forgot to add the output of the perl one liner looks like this so far. 2016-05-07T09:07:42 – Matthew Morcaldi Mar 14 '18 at 5:00
  • You don't need to write single characters as [-], [T] or [:]. Square brackets are meant as character classes, thus make sense if there's 2 or more chars inside. – yahol Mar 14 '18 at 5:55
3

Try this:

perl -nle'($time, $host) = /^(\S+)\s(?:\S+\s+){8}\S+="(\S+?)"/; print "$time $host"'

Output:

2016-05-07T09:07:04.933343+00:00 jamaican.com
  • \S means non-space
  • \s is space
  • (?:) is a logical grouping that's not captured
  • {8} are the skipped "words"
  • \S+="(\S+?)" means: skip until = and capture what's between the two " quotes
  • ($time, $host) = /.../ assigns the two captured groups to $time and $host
0
0

You need to capture the hostname with a second capture group.

e.g. using your sample input:

$ cat test.log 
2016-05-07T09:07:04.933343+00:00 heroku[router]: status=301 bytes=680 service=2698ms connect=1ms dyno=web.2 fwd="10.29.10.29" at=info host="jamaican.com" request_id=32fc8d88-99f8-4cc2-89f9-284d059eebf8 method=GET path="/blog"

This perl one-liner extracts both timestamp and the hostname field. I've improved the regex a little as well, using \d for digits, with a match count for each.

$ perl -lne 'print "$1 $2" if (m/(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d+\+\d{2}:\d{2}) ([^ ]+) /)' test.log
2016-05-07T09:07:04.933343+00:00 heroku[router]:

Another alternative:

$ perl -lne 'print "$1 $2 $3" if (m/(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d+\+\d{2}:\d{2}) ([^[]+)\[([^]]+)\]/)' test.log
2016-05-07T09:07:04.933343+00:00 heroku router

I assumed you wanted hostname of the local machine (I didn't even notice the host="jamaican.com" part of the log entry). That's probably not what you wanted, so if you want the hostname that's inside the double-quotes after host=, then:

$ perl -lne 'print "$1 $2" if (m/(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d+\+\d{2}:\d{2}).*host="([^"]+)"/)' test.log
2016-05-07T09:07:04.933343+00:00 jamaican.com 

or (much simpler):

$ perl -lne 'print "$1 $2" if (m/(^\S+).*host="([^"]+)"/)' test.log
2016-05-07T09:07:04.933343+00:00 jamaican.com

Or extract the timestamp, parse it with Date::Parse and reformat it with Date::Format.

$ perl -MDate::Parse -MDate::Format -lne \
  'if (m/(^\S+).*host="([^"]+)"/) {
     print join(" ", time2str("%Y-%m-%d %R %z",str2time($1)), $2)
   }' test.log
2016-05-07 19:07 +1000 jamaican.com

Note: the timestamp has been converted to local timezone (for me, that's +1000 or Australian Eastern Standard Time). time2str() uses local timezone by default, but you can give it a third argument (time2str(TEMPLATE, TIME [, ZONE])) to make it output the time in any other zone.

$ perl -MDate::Parse -MDate::Format -lne   'if (m/(^\S+).*host="([^"]+)"/) {
     print join(" ", time2str("%Y-%m-%d %R %z",str2time($1),"+0"), $2)
   }' test.log
2016-05-07 09:07 +0000 jamaican.com
2
  • i guess the OP wants the host value.. host="jamaican.com" – Kamaraj Mar 14 '18 at 6:14
  • ok, that's easy. (i didn't even notice the host="..." part of the log entry. i assumed he just wanted the second field) – cas Mar 14 '18 at 8:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.