1

code:

grep -rI --exclude=*_*.log -B20 -A1 "Status:   Error" > /var/log/alertError.log

Input log example:

[06/07/20 20:38:53.911]:loopback ST:                  token-src-name() 

[06/07/20 20:38:53.914]:loopback ST:                    Token Value: "DVADER". 

[06/07/20 20:38:53.916]:loopback ST:                  token-text(",OU=users,O=data") 

[06/07/20 20:38:53.919]:loopback ST:    Arg Value: "CN=DVADER,OU=users,O=data". 

[06/07/20 20:38:53.922]:loopback ST:                description("Removed by Termination Process") 

[06/07/20 20:38:53.926]:loopback ST:             token-text("Removed by Termination Process") 

[06/07/20 20:38:53.929]:loopback ST:                  Arg Value: "Removed by Termination Process". 

[06/07/20 20:38:53.943]:loopback ST: DirXML Log Event -------------------

     Driver:   \StarWars\system\Driver Set\User Processor

     Channel:  Subscriber

     Status:   Error

     Message:  Code(-9217) Error in

This input log shows a section of the log I want to grab. However I want to match on the date for the days I want to search the log. I am using grep as I can do a recursive search of all log files in a structure. My grep returns all of the data I want, except I now want to exclude chunks of code that is older. So the date is not on all lines. The grep is returning a chunk of code with the switches B and A that I need. So if you saw the entire line, the message and status lines are consistent with multiple operations going on. I could grab all types of Message or status values with this grep command, but I just don't know how to then eliminate all of the results to only those chunks of code that are for a given date range.

Improve this question
5
  • Generally, it's inadvisable to parse XML with regular expressions. Please show a sample of your actual log file. – glenn jackman Jun 9 '20 at 16:08
  • modified the post. It is just a log file that has some xml data, The XML data isn't needing to keep any structure with what I am doing and won't be reused. Just gleaning data. – FP55 Jun 9 '20 at 16:43
  • It's unclear for me what you are trying to grep. Could you please describe it more explicit? – danielleontiev Jun 9 '20 at 17:58
  • So my grep is grabbing the "Status: Error" with a number of rows above and below that. That is working fine. What I want to exclude are blocks of code where the dates don't coincide for the day I am wanting to look at. But the date is not included on the same line as the "Status: Error" line. I was hoping I could feed in the block of code that this would be spitting out to the file only if that block of code contained the date I was looking for in that block of code. – FP55 Jun 9 '20 at 18:15
  • Is log example your sample input? If so please edit your question to show the expected output given that input. Use find to find files and grep to g/re/p files to keep your life and code simple, i.e. don't use grep to find files. There is a big clue in the name of each tool what it's supposed to be used for! In this case though it's extremely unlikely that grep will be adequate since you're not trying to just do g/re/p, you'll probably need awk or similar. – Ed Morton Jun 9 '20 at 20:34
0

Here is the python solution:

with open("log.txt") as f:                                  # open file log.txt
    lines = f.readlines()                                   # load lines
    nonempty = filter(lambda x: x.strip() != "", lines)     # filter out empty lines
    newlines = []                                           # list for out result lines
    for l in nonempty:                                      # iterate lines
        l = l.rstrip()                                      # cut '\n' from the right of each line
        last_idx = len(newlines) - 1                        # index of the last element in the list
        if l.startswith(" "):                               # this lines are lines with your traceback
            newlines[last_idx] += l                         # add to the "normal" log element
        else:                                               # this are "normal" elements
            newlines.append(l)                              # add them to the list

    print("\n".join(newlines))                              # create output and print to stdout

This output contains "Status" and "Date" on the same line and you could grep it

Place it (say normalize.py) near you log file (say log.txt) and run with python3 normalize.py

Improve this answer
0

Your question isn't clear but is this what you're trying to do?

$ awk -v tgt='06/07/20' '
    /^\[/ { prt() }
    NF { rec = rec $0 ORS }
    END { prt() }

    function prt() {
        if ( index(rec,tgt) == 2 ) {
            printf "%s", rec
        }
        rec = ""
    }
' file
[06/07/20 20:38:53.911]:loopback ST:                  token-src-name()
[06/07/20 20:38:53.914]:loopback ST:                    Token Value: "DVADER".
[06/07/20 20:38:53.916]:loopback ST:                  token-text(",OU=users,O=data")
[06/07/20 20:38:53.919]:loopback ST:    Arg Value: "CN=DVADER,OU=users,O=data".
[06/07/20 20:38:53.922]:loopback ST:                description("Removed by Termination Process")
[06/07/20 20:38:53.926]:loopback ST:             token-text("Removed by Termination Process")
[06/07/20 20:38:53.929]:loopback ST:                  Arg Value: "Removed by Termination Process".
[06/07/20 20:38:53.943]:loopback ST: DirXML Log Event -------------------
     Driver:   \StarWars\system\Driver Set\User Processor
     Channel:  Subscriber
     Status:   Error
     Message:  Code(-9217) Error in

or maybe this?

$ awk -v tgt='06/07/20' '
    /^\[/ { prt() }
    NF { rec = rec $0 ORS }
    END { prt() }

    function prt() {
        if ( (index(rec,tgt) == 2) && (rec ~ /Status:[[:space:]]+Error/) ) {
            printf "%s", rec
        }
        rec = ""
    }
' file
[06/07/20 20:38:53.943]:loopback ST: DirXML Log Event -------------------
     Driver:   \StarWars\system\Driver Set\User Processor
     Channel:  Subscriber
     Status:   Error
     Message:  Code(-9217) Error in

You can call awk from find easily, e.g.:

find . -type f -name '*.log' -exec awk '....' {} +
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.