4

I am new to bash scripting. Could someone help me with the following? I have a log file with output as shown below.

I'm trying to grep for lines of an output with logDurationMillis>=950ms

logAlias:Overall,logDurationMillis:382,logTimeStart:2019-07-24_15:30:06.075,logTimeStop:2019-07-24_15:30:06.107
logAlias:Overall,logDurationMillis:388,logTimeStart:2019-07-24_15:30:06.406,logTimeStop:2019-07-24_15:30:06.444
logAlias:Overall,logDurationMillis:545,logTimeStart:2019-07-24_15:30:06.583,logTimeStop:2019-07-24_15:30:06.638
logAlias:Overall,logDurationMillis:961,logTimeStart:2019-07-24_15:30:06.599,logTimeStop:2019-07-24_15:30:06.660
logAlias:Overall,logDurationMillis:640,logTimeStart:2019-07-24_15:30:07.197,logTimeStop:2019-07-24_15:30:07.237
logAlias:Overall,logDurationMillis:934,logTimeStart:2019-07-24_15:30:07.474,logTimeStop:2019-07-24_15:30:07.508
logAlias:Overall,logDurationMillis:336,logTimeStart:2019-07-24_15:30:07.546,logTimeStop:2019-07-24_15:30:07.582

The values are always in the second comma-delimited column.

11

With awk:

  1. if you know "logDurationMillis" is the second item:

    awk -F'[:,]' -v limit=950 '$4 >= limit' file
    
  2. otherwise

    awk -F'[:,]' -v limit=950 '{
        for (i=1; i<NF; i+=2) 
            if ($i == "logDurationMillis" && $(i+1) >= limit) 
                print
    }' file
    
  • 2
    I did not know you could do that with the delimiter! Awesome – Jesse_b Jul 24 at 19:42
  • logDurationMillis is the second field so i tried the 2nd script you have provided, It worked.. Thank you so much – Aslam534 Jul 24 at 19:52
  • @Jesse_b: awk (even regular awk) can accept regexps as delimiter, and also extended ones (ex: -F 'this|that[ _]*also' will separate fields with "this" and with "thatalso" or "that also" or "that_also" (and allow even multiple occurences of space or _, as there is a '*' in the regexp). It can be really handy. Here, in this answer, it separates fields with either 1 "," or 1 ":", so it will separate fields at each occurence of "," or ":" . (if the line starts with a separator, $1 is empty and $2 is the next field) – Olivier Dulac Jul 25 at 18:05
  • @glennjackman: Could you please edit that first one? to put $4 as you clearly intended to. (I am not sure if it is correct etiquette to edit it myself as it "changes" the answer...) – Olivier Dulac Jul 25 at 18:09
  • I did update it. It would be absolutely appropriate for you to edit my answer. The OP changed the input data after I answered, and a simple change like that does not alter the intent of the answer. – glenn jackman Jul 25 at 19:44
5

Assuming your file is named logFile, this command will do it:

egrep ',logDurationMillis:(9[5-9][0-9]|[1-9][0-9]{3,}),' logFile
  • Using a regexp for a numeric comparison results in hard to understand/maintain code. To compare numbers, just use awk with a numeric comparison - there's zero reason to do this with regexps in grep. – Ed Morton Jul 25 at 17:12
1

another awk one:

awk -F':|,' '$4 > 950' file

prints:

logAlias:Overall,logDurationMillis:961,logTimeStart:2019-07-24_15:30:06.599,logTimeStop:2019-07-24_15:30:06.660

Update (due to OPs Question:)

you can reconstruct the fields like this:

awk -F':|,' '$4 > 950 {print $3 ":" $4 "," $5 ":" $6 ":" $7 ":" $8 }' file

prints:

logDurationMillis:961,logTimeStart:2019-07-24_15:30:06.599

(there is probably a simpler way that concatinates the fields 3-8, but you also need to reconstruct the different field separators)

  • 1
    (1) This is basically the same as glenn jackman’s answer. (2) The question says “>=950”, but your answer says > 950. – G-Man Jul 25 at 2:58
  • 2
    That double pipe || looks odd. The intervening part would match the empty string too, but what use is that? Why not just :|,? – ilkkachu Jul 25 at 7:10
  • 1
    @ilkkachu:  Good catch; I missed that.  I suspect that nath meant :|, — as I said, equivalent to glenn’s [:,] — but reflexively typed || for “OR”.  But, AFAICT in six to eight minutes of testing, this answer works.  Can you demonstrate that the extra | actually causes it to fail? – G-Man Jul 25 at 13:24
  • 1
    @G-Man a | immediately following | is undefined behavior per POSIX, see pubs.opengroup.org/onlinepubs/9699919799/basedefs/…: | ... A <vertical-line> appearing ... immediately following a <vertical-line> ... produces undefined results.. So that code could do anything in any given awk. – Ed Morton Jul 25 at 17:16
  • Thanks Nath, 'awk -F':||,' '$4 > 950' file' works :) Actual log file is like logAlias:Overall,logDurationMillis:382,logTimeStart:2019-07-24_15:30:06.075,logTimeStop:2019-07-24_15:30:06.107, UniqueId.... question :How do i get only following fields in result logDurationMillis:961,logTimeStart:2019-07-24_15:30:06.075,UniqueId – Aslam534 Jul 26 at 6:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.