I'm trying to extract all logs between two time-stamps. Some lines may not have the time-stamp, but I want those lines to be included - I want every line that falls under two time stamps to be included in the extracted logs. the extracted logs should contain the 1st and the last timestamp lines as well.

Note: the start time-stamp or end time-stamp may not be there in the log, but I want every line between these two time stamps to be extracted.

My log time-stamp structure looks like: 25-01-2017 07:06:16:860

The awk command I've written is fetching the lines which contains timestamp only and skipping all other lines and its fetching logs end_time-1 i.e. if I've giving end_time as 11:30 so its fetching logs till 11:29 or sometimes very strange less than that too.

PFB the command I've written:

awk -v date=${date} \
    -v start_time=${start_time} \
    -v end_time=${end_time} \
'{if (($0 >= date FS start_time) && ($0 <= date FS end_time)) print $0; }' \
     $log.$server_name.log > $requester_email.log 

Please note that this command will be inside a script where I'm asking the user to just enter the details manually so when I'm asking the start and end time then user have to enter in this format HH:MM i.e. for above timestamp the user will enter time something like 07:06

Will really appreciate if anyone can help me out here please.

Log Example:

25-01-2017 07:23:51:772 [DEBUG] - sdsdsadadjhadsjhasdjhajhdahdkjhadjkhasjkdhjhg asdgahdgasdhghasdghagdshdhasgadahdghasdgasd
25-01-2017 07:23:51:772 [DEBUG] - asdasd.asdasd.asdasdas.asdasd.asdasda.dfsfd.cxzzxczxczcwdqweqwe.adadsasd.asdasdasdadasdadasd.adadasdasdasd.as: adasdasdadadasdsdfsdfsdfsfssdf..<Request xmlns="adasdadasdasdasdasdasdasadadasd" xmlns:xsi="http://www.w3.org/2001/">
    <Attribute AttributeId="asdasdasasdasdasdasdasdasdasdasddaasdasdasdasdaas" DataType="http://www.w3.org/2001/XMLSchema#string">
    <!--Check something somthing-->
    AttributeId="asdasdasasdasdasdasdasdasdasdasddaasdasdasdasdaas" DataType="http://www.w3.org/2001/XMLSchema#string">
    <Attribute AttributeId="sdfsdfsdsdfsdfsdfsdfsdfsfxcvxvxcvvxvcxvwerqwr" DataType="http://www.w3.org/2001/XMLSchema#string">
25-01-2017 07:23:51:775 [DEBUG] - sdsdsadadjhadsjhasdjhajhdahdkjhadjkhasjkdhjhg asdgahdgasdhghasdghagdshdhasgadahdghasdgasd
  • 1
    Welcome to stackexchange. Please update question to include example of typical input and desired output.
    – steve
    Jan 27, 2017 at 16:59
  • thanks steve, this command will be inside a script where a user friendly message will be displayed to enter date, start and end time, which will be stored in the variables mentioned in awk command i.e. date, start_time, end_time. And output should be like it should fetch all the logs line between the date start_time and date end_time. Some lines may not have timestamp but those also should be included.
    – Vaibhav
    Jan 27, 2017 at 17:03

1 Answer 1


Your approach has two problems:

  1. You compare the whole input line with the timestamp lexicographically. If the line starts with the end time stamp and contains additional characters, it will be sorted after the time stamp, as longer strings are sorted after shorter strings.
  2. You have no rule that handles lines that do not start with a time stamp. As they are not sorted between the start and end time stamps, they will not be included in the output. How are these lines identified?

Try something like this:

$1 == date {
    start = substr($2, 0, length(start_time));
    end = substr($2, 0, length(end_time));
    if (start_time <= start && end <= end_time) print;
{ print; }
  • Thanks for your comment Michael. Can you please help with the code ?
    – Vaibhav
    Jan 30, 2017 at 10:44
  • @Vaibhav It would be easier if you could explain how to identify lines that do not start with a timestamp. Do they start with whitespace, for example? Jan 30, 2017 at 12:21
  • I've updated my question with a sample from my logs. Which I think will give you an idea. As per my current script, it is fetching the lines with timestamp only and not the line <Resource> onwards. Secondly, it will fetch only until end_time-1;
    – Vaibhav
    Jan 30, 2017 at 12:52
  • its not working... $1 == date { start = substr($2, 0, length(start_time)); end = substr($2, 0, length(end_time)); if (start_time <= start && end <= end_time) print; next; } <server.log.name>
    – Vaibhav
    Jan 31, 2017 at 11:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .