3

Completely new to scripting in general, and have been working on a script to go through our logs which contain entries for a variety of different things. What I'm interested in, is the entries that log user activity (logins, clicks, etc on our site).

I've pieced together the following awk script (with a ton of help) which almost works perfectly, except it prints output for lines that don't contain a username. I'd like to exclude them, and haven't been able to find out how to do that (I assume it's something simple).

Here is the code:

awk '
{       split($3, d, "@")
        match($0, /"username":"[^"]*"/)
        user = substr($0, RSTART + 12, RLENGTH - 13)
        c[d[1] OFS user]++
}
END {   for(i in c)
                printf("%4d %s\n", c[i], i)
}' mycompany.log | sort -k3,3 -k2,2

I'm just not sure how to exclude all lines that do not contain username from the output.

Here is a sample of a line that contains a username:

qtp111659197-5776 - 05-26@09:37:34:240 INFO  (TimingInfoProxy.java:41)     - com.mycompany.api.ApiHandler-0>getUniqueDataBySource(data,{"has_values":false,"last_event_triggered":"","user_info":{"username":"joe@mycompany.com","orgid":"69d467a7-9786-47e1-9c12-bb40f9bfc65d","ip":"127.0.0.1"},"date_range":{"min_date":"","start_date":"","end_date":"","trending_start_date":"","trending_end_date":""},"terms":{"and_filtering":[]}},) 

And here's a short example of a line that doesn't:

main - 05-22@10:05:21:387 INFO  (ContextLoader.java:313)     - Root WebApplicationContext: initialization completed in 9519 ms
  • In what way exactly do they 'not contain' a username? is the "username": keystring missing, or is there an empty quoted value like "username":""? or something else? Please edit your post to include sample entries both with and without the username. – steeldriver May 28 '15 at 16:37
  • I edited it to include samples, but basically yes the "username": keystring is missing in lines that don't include usernames. – zoo_live_crew May 28 '15 at 16:54
  • So basically the match part of the code tells it to find the regexp "username":"[^"]*" which in terms of our logs is username@mycompany.com Isn't there something simple I can put in there that basically tells it not to return any output if it can't find regexp "username":"[^"]*" in any given line of input? (sorry I'm very new to this) – zoo_live_crew May 28 '15 at 22:20
1

Since you're already piping to other shell commands, I'd just use grep.

awk '
{
...
}' mycompany.log | grep -f '"username:"' | sort -k3,3 -k2,2

(Note that I'm using -f because the username tag never varies.)

Edit: that's the most obvious way. An arguably "better" way would be to use awk itself, e.g.

awk '
/\"username\":/ {
...
}' mycompany.log | sort -k3,3 -k2,2
0

Your code isn't checking the result of match. The following fixes it.

RSTART is set by invoking the match function. Its value is the position of the string where the matched substring starts, or zero if no match was found.

if(RSTART){
   user = substr($0, RSTART + 12, RLENGTH - 13)
   c[d[1] OFS user]++
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.