2

My system date format:

Thu Jun 13 12:55:18 EDT 2019

My log format:

193.118.26.141 - - [01/May/2019:00:08:49 -0400] "GET / HTTP/1.1" 200 27

Can someone help me, how to get last 5 minutes of logs please?

3
0

Since you will need extensive text processing and time manipulations, this should be done using awk, but will still need the external date command to work.

The following awk program (let's call it parse_log_range.awk) will work:

#!/bin/awk -f
BEGIN{
    "date +%s" | getline now
    close("date +%s")
}

{
    n=match($0,/\[[^]]+\]/)
    tst=substr($0,n+1,RLENGTH-2)

    gsub(/\//," ",tst)  # replace all '/'
    sub(/:/," ",tst)    # replace first ':'
    
    cmd="date +%s -d \"" tst "\""
    cmd | getline tst_s; close(cmd)

    if (now-tst_s<max_age) print
}

In the beginning, it will launch the command date +%s in the shell to query the current time and store it in a variable now.

For each line of the log file, this will

  • extract the timestamp as being the element in between [ ... ]
  • remove the / and the first : to make the format understandable to date
  • create a shell command of the form date +%s -d edited timestamp which will convert the timestamp into "seconds since the epoch"
  • run this command in the shell, and read the result into a variable tst_s

It will then check if the timestamp is less than max_age away from now, where max_age is a timespan you have to specify (in your example, 5 minutes = 300 seconds).

Run the command as

awk -v max_age=300 -f parse_log_range.awk logfile.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.