4
2021-02-10 08:17:38 dell AptDaemon.Worker: INFO: Processing transaction /org/debian/apt/transaction/08a244f7b8ce4fad9f6b304aca9eae7a
2021-02-10 08:17:50 dell AptDaemon.Worker: INFO: Finished transaction /org/debian/apt/transaction/08a244f7b8ce4fad9f6b304aca9eae7a
2021-02-10 08:18:50 dell AptDaemon.PackageKit: INFO: Initializing PackageKit transaction
2021-02-10 08:18:50 dell AptDaemon.Worker: INFO: Simulating trans: /org/debian/apt/transaction/37c3ef54a6ba4933a561c49b3fac5f6e
2021-02-10 08:18:50 dell AptDaemon.Worker: INFO: Processing transaction /org/debian/apt/transaction/37c3ef54a6ba4933a561c49b3fac5f6e
2021-02-10 08:18:51 dell AptDaemon.PackageKit: INFO: Get updates()
2021-02-10 08:18:52 dell AptDaemon.Worker: INFO: Finished transaction /org/debian/apt/transaction/37c3ef54a6ba4933a561c49b3fac5f6e
2021-02-10 08:24:36 dell AptDaemon: INFO: Quitting due to inactivity
2021-02-10 10:17:38 dell AptDaemon.Worker: INFO: Processing transaction /org/debian/apt/transaction/08a244f7b8ce4fad9f6b304aca9eae7a
2021-02-10 10:17:50 dell AptDaemon.Worker: INFO: Finished transaction /org/debian/apt/transaction/08a244f7b8ce4fad9f6b304aca9eae7a
2021-02-10 10:18:50 dell AptDaemon.PackageKit: INFO: Initializing PackageKit transaction
2021-02-10 10:18:50 dell AptDaemon.Worker: INFO: Simulating trans: /org/debian/apt/transaction/37c3ef54a6ba4933a561c49b3fac5f6e
2021-02-10 10:18:50 dell AptDaemon.Worker: INFO: Processing transaction /org/debian/apt/transaction/37c3ef54a6ba4933a561c49b3fac5f6e
2021-02-10 10:18:51 dell AptDaemon.PackageKit: INFO: Get updates()
2021-02-10 10:18:52 dell AptDaemon.Worker: INFO: Finished transaction /org/debian/apt/transaction/37c3ef54a6ba4933a561c49b3fac5f6e
2021-02-10 10:24:36 dell AptDaemon: INFO: Quitting due to inactivity

would like to grep logs between 2021-02-10 08:00:00 and 2021-02-10 09:00:00

 sudo sed -n '/2021-02-10 08:00:00/,/2021-02-10 09:00:00/p' /var/log/app.log

This command doen't print any output. where as If I use exact timestamps then it's printing output. For example If I use below command prints the output

sudo sed -n '/2021-02-10 08:17:38/,/2021-02-10 08:24:36/p' /var/log/app.log

please suggest how to use sed command to grep between timestamps, if log file doesn't contain starting timestamp and ending timestamp but contains logs between timestamps.

2

2 Answers 2

7

that construct sed '/start-date/,/end-date/' is not about defining the range; that will only return lines start from a line that contains "start-date" up-to first line containing "end-date" not the date-range.

Use sed as following:

sed -E '/2021-02-10 (08:[0-5][0-9]:[0-5][0-9]|09:00:00)/!d' /log/infile

same with grep:

grep -E '2021-02-10 (08:[0-5][0-9]:[0-5][0-9]|09:00:00)' /log/infile

or with awk:

awk '/2021-02-10 (08:[0-5][0-9]:[0-5][0-9]|09:00:00)/' /log/infile

specifically for your need.


Using awk for a solution to handling the range of dates correctly and calling external command GNU date for date conversation from human readable to epoch time:

awk -v start='2021-02-10 08:00:00' -v end='2021-02-10 08:24:36' -v q="'" '
BEGIN{
    st="date -d" q start q " +%s"; st |getline start; close(st);
    ed="date -d" q end   q " +%s"; ed |getline end;   close(ed)
}
{ dt=$1" "$2; epoch="date -d" q dt q " +%s"; epoch |getline dt; close(epoch) };
(dt>=start && dt<=end)' infile

Or using GNU awk instead for the date handling using the mktime() function:

awk -v start='2021-02-10 08:00:00' -v end='2021-02-10 08:24:36' '
 BEGIN{ gsub(/[:-]/," ", start); gsub(/[:-]/," ", end) }
      { dt=$1" "$2; gsub(/[:-]/," ", dt) }
 mktime(dt)>=mktime(start) && mktime(dt)<=mktime(end)' infile

Note that we replaced all colons and dashes to space as mktime() accept the time in the format of YYYY MM DD HH MM SS.

7
  • The above solution works fine for time period 8:00 to 9:00. But if I want to grep between 8:30 to 10:30 how to it? I have used sed -E '/2021-02-10 08:3[0-9]:[0-5][0-9]|2021-02-10 10:30:00/!d' but it's not working.
    – bhr
    Feb 18, 2021 at 10:05
  • 1
    @bhr you want sed solution but sed is not the right tool for that, since for every range you must manually adjust the regex for that; and another solutons based on the range already mentioned in the Q/A I flagged under your question and I don't want duplicate them here; for that range you need ` sed -E '/2021-02-10 (08:3[0-9]:[0-5][0-9]|09:[0-5][0-9]:[0-5][0-9]|10:[0-2][0-9]:[0-5][0-9]|10:30:00:00)/!d'` Feb 18, 2021 at 10:19
  • if I use instead of grep or awk, how can I achive it... Is there any generic way to without adding ranges like [0-5][0-9]... since my search will keep on changing, so I would like to prepare script like start_time=$1, end_time=$2, awk '/$start_time|$end_time' /log/infile
    – bhr
    Feb 18, 2021 at 10:35
  • @bhr see the awk solution added; you now just need to update -v start='....' and -v end='...' parts as start and end dates or replace with shell variables like -v start="$start_time" -v end="$end_time" ... Feb 18, 2021 at 10:56
  • If your system has journalctl, its --since= and --until= would help.
    – waltinator
    Feb 19, 2021 at 2:20
0

If anybody wants to get the logs within the particular time rage, maybe you want to get the logs within 5 min you can do like the below

For dynamic time range (eg you dont know the time range but needs logs between last 5 min)

#!/bin/bash
LOG_FILE="./modsec.txt"
SEARCH_WORD="ModSecurity: Access denied"
START_TIME=$(date -d "5 minutes ago" "+%H:%M")
END_TIME=$(date "+%H:%M")

# using sed
sed -n "/"$START_TIME"/,/"$CURRENT_TIME"/p" "$LOG_FILE" | grep -w "$SEARCH_WORD"

For static time range (you know the time range)

sed -n "/hh:mm/,/hh:mm/p" "$LOG_FILE" | grep -w "$SEARCH_WORD" (you can replace the time range based on your timestamp)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .