0

I'm trying to figure out a logic where we can pass time & date range as input parameter that run it against log file like below and then extract matching patterns from each block of text between blank lines:

Here's a sample content in a log file:

# 10/Oct/2017:11:14:12 -0400; conn=-1; op=59
dn: dc=capitalone,dc=com
changetype: modify
replace: ds-sync-state
ds-sync-state: 0000015b51d7bf33057e00000002
ds-sync-state: 0000015b53b701d30f0c00000003
ds-sync-state: 0000015b51d655e11a0c00000005

# 11/Oct/2017:10:40:29 -0400; conn=-1; op=44
dn: cn=schema
changetype: modify
replace: ds-sync-generation-id
ds-sync-generation-id: 8408

# 11/Oct/2017:10:40:30 -0400; conn=-1; op=59
dn: cn=admin data
changetype: modify
replace: ds-sync-state
ds-sync-state: 0000015b50e62ee409b700000002
ds-sync-state: 0000015b4a174d3f217500000002
ds-sync-state: 0000015b4ed4fcfb23a500000008
3
  • I'm not 100% sure what you want to do. Is that sample, a single log file or multiple log files? What data do you want to extract? Are there many log files with the same format? Should 11/Oct/2017:10:40:30 be treated differently to 11/Oct/2017:10:40:29? Please expand your question for clarity.
    – Tigger
    Commented Oct 13, 2017 at 2:18
  • There are multiple logs starting with same name (e.g. access_1, access_2). Yes, the sample is from a single log. Based on the date & time range we pass it needs to grep below attributes from the block of text under found date & time. Would like to pass the range in this way: "--start 11/Oct/2017:10:40:29 --end 11/Oct/2017:10:41:00": Grep for following attributes: dn: changetype: replace:
    – Riz
    Commented Oct 13, 2017 at 3:05
  • Your comment response is still not very clear. For example, are the --start and --end values "fuzzy" or "exact" for what is in the log files? Using your example (in the comment) and the sample log file data in the question, no records would be matched (exact) OR the last two would be matched (fuzzy).
    – Tigger
    Commented Oct 13, 2017 at 4:05

1 Answer 1

1

GNU awk solution:

lookup_by_daterange.awk script:

#!/usr/bin/awk -f

function get_ts(dt, ts){ 
    gsub("/"," ",dt); 
    sub(":"," ", dt);
    cmd="date -d \""dt"\" +%s"; 
    while(cmd | getline t) ts=t; 
    close(cmd); 
    return ts 
}
BEGIN {
    from = get_ts(from);  # `from` datetime
    to = get_ts(to);    # `to` datetime
}
/^# [0-9]{2}\//{ 
    ts = get_ts($2); 
    if (ts >= from && ts <= to) { 
        f=1; n=NR+3 
    } 
}
f && NR<=n { 
    print 
}
NR==n { 
    print "" 
}

Usage:

awk -v from="11/Oct/2017:10:40:29" -v to="11/Oct/2017:10:41:00" -f lookup_by_daterange.awk logfile

The output:

# 11/Oct/2017:10:40:29 -0400; conn=-1; op=44
dn: cn=schema
changetype: modify
replace: ds-sync-generation-id

# 11/Oct/2017:10:40:30 -0400; conn=-1; op=59
dn: cn=admin data
changetype: modify
replace: ds-sync-state
2
  • Thanks bunch Roman... the script you wrote works like a champ... this captures the date and next three lines... is it possible to provide what parameter to return instead. For instance sometimes I only need one of the attributes like; "replace: " to return.
    – Riz
    Commented Oct 13, 2017 at 21:10
  • @Riz, to print only replace: ... lines - replace the line f && NR<=n { with f && NR==n {. That's all Commented Oct 13, 2017 at 21:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .