Fetch the series of logs in between date range when the range contains specific string(s)

Question

I am searching for multiple range of lines from a file with in a given date range when the search string(s) exists. I am not an Unix expert, don't know how to achieve this. After googling I was able to display the logs in date range by using awk and sed commands (mentioned below), but not able to add a search criteria. Below is the logs file format:

Log file content

[2020/07/05 21:10:28.961][INFO ][ABCDDDDDDD]
------------------------------------------------------------
ID: ABCDDGSDFEWRER
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9789,
  "event" : "something",
  COMMON_TEXT: COMMON
    "valid" : true
}
************************************************************
[2020/07/05 21:10:28.816][INFO ][ABCDDDDDEEEEEEE]
------------------------------------------------------------
ID: ABCDDSDFSDFSDFSDFSDFSDFSDFFEWRER
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9089,
  "event" : "One More thing",
  COMMON_TEXT: COMMON
    "valid" : true
}
************************************************************
[2020/07/05 21:10:43.816][INFO ][ABCDDDDDEEFFFFFFFFFFFFFEEE]
------------------------------------------------------------
ID: QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9789,
  "event" : "Second thing",
  COMMON_TEXT: COMMON
    "valid" : False
}
************************************************************

Commands I am using currently:

• awk: (to fetch exact start time since sed command is not recognizing exact start time)

  awk 'substr($1,2,11)>="2020\/07\/05" && substr($1,2,11)<="2020\/07\/05" && substr($2,1,8)>="21:10:28" && substr($2,1,8)<="21:10:43" {print $0}' logfileName
  

• sed: to get the logs (blocks in between two dates) in a given time range. Middle block is not required to be printed (since it does not contains 9789 though in my time range)

  sed -n '/2020\/07\/05 21:10:28.010/,/2020\/07\/05 21:10:30.668/{;p};/2020\/07\/05 21:10:30.668/,/\*\*\*/{;p}' logfileName
  

Here is the sample output I am looking for: 1. The below two blocks are in my desired time limit 2. And it contains "BC" : 9789 (my desired number) Currently I am achieving using above two commands combining and handling the validation (9789) by removing the middle block using java code.

[2020/07/05 21:10:28.961][INFO ][ABCDDDDDDD]
------------------------------------------------------------
ID: ABCDDGSDFEWRER
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9789,
  "event" : "something",
  COMMON_TEXT: COMMON
    "valid" : true
}
************************************************************
[2020/07/05 21:10:43.816][INFO ][ABCDDDDDEEFFFFFFFFFFFFFEEE]
------------------------------------------------------------
ID: QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ
MESSAGE: Event Message received.
------------------------------------------------------------
CONTEXT: {
  "ID" : 1,
  "BC" : 9789,
  "event" : "Second thing",
  COMMON_TEXT: COMMON
    "valid" : False
}
************************************************************

Answer 1

This is somewhat verbose, because:

.. It exposes the variables suitably for making them arguments to the script, assuming this is not a one-off.

.. It is readable and adaptable.

.. It shows the entire block for a log entry, from the timestamp down to the next line consisting of all asterisks.

#! /bin/bash

Low='2020/07/05 21:10:28'
End='2020/07/05 21:10:43'

AWK='
BEGIN { 
    reTS = "[[]20../../.. ..:..:..[.]...[]]";
    reStop = "^[*]+$";
}
function Range (ts) {
    return ((ts < Low || ts > End) ? "N" : "Y");
}
match ($0, reTS) { Block = Range( substr ($0, RSTART+1, RLENGTH-6)); }
Block == "Y" { print; }
$0 ~ reStop { Block = "N"; }
'
    awk -v Low="${Low}" -v End="${End}" "${AWK}" myLog.txt

Answer 2

Just slight change in you sed command

sed -n '/2020\/07\/05 21:10:28\.010/,/2020\/07\/05 21:10:30\.668/{;p};/2020\/07\/05 21:10:30\.668/,/\*\*\*/{;p}'

Your command just needed escaping . by placing backslash preceeding it. i.e, 2020/07/05 21:10:28\.010