0

I am searching for multiple range of lines from a file with in a given date range when the search string(s) exists. I am not an Unix expert, don't know how to achieve this. After googling I was able to display the logs in date range by using awk and sed commands (mentioned below), but not able to add a search criteria. Below is the logs file format:

Log file content

[2020/07/05 21:10:28.961][INFO ][ABCDDDDDDD]
------------------------------------------------------------
ID: ABCDDGSDFEWRER
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9789,
  "event" : "something",
  COMMON_TEXT: COMMON
    "valid" : true
}
************************************************************
[2020/07/05 21:10:28.816][INFO ][ABCDDDDDEEEEEEE]
------------------------------------------------------------
ID: ABCDDSDFSDFSDFSDFSDFSDFSDFFEWRER
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9089,
  "event" : "One More thing",
  COMMON_TEXT: COMMON
    "valid" : true
}
************************************************************
[2020/07/05 21:10:43.816][INFO ][ABCDDDDDEEFFFFFFFFFFFFFEEE]
------------------------------------------------------------
ID: QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9789,
  "event" : "Second thing",
  COMMON_TEXT: COMMON
    "valid" : False
}
************************************************************

Commands I am using currently:

  • awk: (to fetch exact start time since sed command is not recognizing exact start time)
    awk 'substr($1,2,11)>="2020\/07\/05" && substr($1,2,11)<="2020\/07\/05" && substr($2,1,8)>="21:10:28" && substr($2,1,8)<="21:10:43" {print $0}' logfileName
    
  • sed: to get the logs (blocks in between two dates) in a given time range. Middle block is not required to be printed (since it does not contains 9789 though in my time range)
    sed -n '/2020\/07\/05 21:10:28.010/,/2020\/07\/05 21:10:30.668/{;p};/2020\/07\/05 21:10:30.668/,/\*\*\*/{;p}' logfileName
    

Here is the sample output I am looking for: 1. The below two blocks are in my desired time limit 2. And it contains "BC" : 9789 (my desired number) Currently I am achieving using above two commands combining and handling the validation (9789) by removing the middle block using java code.

[2020/07/05 21:10:28.961][INFO ][ABCDDDDDDD]
------------------------------------------------------------
ID: ABCDDGSDFEWRER
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9789,
  "event" : "something",
  COMMON_TEXT: COMMON
    "valid" : true
}
************************************************************
[2020/07/05 21:10:43.816][INFO ][ABCDDDDDEEFFFFFFFFFFFFFEEE]
------------------------------------------------------------
ID: QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9789,
  "event" : "Second thing",
  COMMON_TEXT: COMMON
    "valid" : False
}
************************************************************
6
  • for the awk command, use "2020/07/05" instead of "2020\/07\/05" ... and you could use =="2020/07/05" if your upper/lower checks are same value
    – Sundeep
    Jul 6, 2020 at 12:29
  • Do you need the whole 13-line block for each entry? Your current code would only show the identified lines themselves. Given the format shown, briefer matches are available in awk. Jul 6, 2020 at 12:35
  • edit your question to show the exact expected output given your posted sample input. Otherwise we're just guessing at your requirements. Your sample input shows some lines indented several spaces and others flush up against the start of the line - if that's not how it really is then fix it.
    – Ed Morton
    Jul 6, 2020 at 21:42
  • @nag please see unix.stackexchange.com/help/merging-accounts for help on merging your two accounts so you can recover this question.
    – terdon on strike
    Jul 7, 2020 at 9:00
  • Thanks @terdon, I am able to merge my accounts now. and thanks others for replying to the post. I am not having trouble in access the dates. Facing difficulties in fetching the logs in desired range along with the search condition. Updating the question in more readable form
    – nag
    Jul 7, 2020 at 16:34

2 Answers 2

1

This is somewhat verbose, because:

.. It exposes the variables suitably for making them arguments to the script, assuming this is not a one-off.

.. It is readable and adaptable.

.. It shows the entire block for a log entry, from the timestamp down to the next line consisting of all asterisks.

#! /bin/bash

Low='2020/07/05 21:10:28'
End='2020/07/05 21:10:43'

AWK='
BEGIN { 
    reTS = "[[]20../../.. ..:..:..[.]...[]]";
    reStop = "^[*]+$";
}
function Range (ts) {
    return ((ts < Low || ts > End) ? "N" : "Y");
}
match ($0, reTS) { Block = Range( substr ($0, RSTART+1, RLENGTH-6)); }
Block == "Y" { print; }
$0 ~ reStop { Block = "N"; }
'
    awk -v Low="${Low}" -v End="${End}" "${AWK}" myLog.txt
0

Just slight change in you sed command

sed -n '/2020\/07\/05 21:10:28\.010/,/2020\/07\/05 21:10:30\.668/{;p};/2020\/07\/05 21:10:30\.668/,/\*\*\*/{;p}'

Your command just needed escaping . by placing backslash preceeding it. i.e, 2020/07/05 21:10:28\.010

1
  • Thanks for your reply. as updated in the question I am getting the logs and no issue with the dates, looking help in including additional conditions to the above command along with the range
    – nag
    Jul 8, 2020 at 2:57

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .