1

I want to get some parts from my log file, I tried to cut request part for to get the user, module, action, doAjax and ajaxAction

For example, I have

195.xx.x.x - - [25/Apr/2017:09:60:xx +0200] "POST /userx/index.php?m=contacts&a=form&...
192.xx.x.x - - [25/Apr/2017:09:45:xx +0200] "POST /usery/index.php?m=customer&doajax=request&action=getContacts...
197.xx.x.x - - [25/Apr/2017:09:20:xx +0200] "GET /userx/index.php?m=meeting&doajax=date&id=3

and I want to have:

[user]|[module]|[action]|[doAjax]|[ajaxAction] 
usery  contacts  form     null     null
userx  customer  null     request  getContacts
userz  meeting   null     date     null

Where:

userx --> user  
m=xxx -->module   
a=xxx -->action  
doajax=xxx-->doAjax   
action=xxx-->ajaxAction  

I tried to use awk, set but for to cut just the 7th column where I can find my request with this command:

awk '{printf $7; next ; }' logfile

So How can I do for to extract the user, module, action, doAjax and ajaxAction after to print just my request?

2

A perl "one-liner":

$ perl -lne '
BEGIN{
    printf "%-10s%-10s%-10s%-10s%-15s\n", qw([user] [module] [action] [doAjax] [ajaxAction]);
} 
$usr = $mde = $act = $doAj = $ajAc = "null"; 
$usr=$1 if m|\s/([^/]+)/|; 
$mde=$1 if /m=(.+?)(&|$)/; 
$act=$1 if /a=(.+?)(&|$)/; 
$doAj=$1 if /doajax=(.+?)(&|$)/; 
$ajAc=$1 if /action=(.+?)(&|$)/; 
printf "%-10s%-10s%-10s%-10s%-15s\n", ($usr,$mde,$act,$doAj,$ajAc)' file 
[user]    [module]  [action]  [doAjax]  [ajaxAction]   
userx     contacts  form      null      null           
usery     customer  null      request   getContacts    
userx     meeting   null      date      null           

The basic trick here is to search for each of the strings identifying your URL parts and, if found, set the corresponding variable to it. In each case, we look for the identifier followed by an = (e.g. m=) and then either a & or the end of the line (&|$). Because the matched portion is put in a parenthesis (e.g. m=(.+?)), we can then refer to it as $2 and that's what's saved in each variable.

If you really need to have | as a separator, and don't object to the fact that it will make the output less readable, you can use this instead:

$ perl -lne '
BEGIN{
    printf "%s|%s|%s|%s|%s\n", qw([user] [module] [action] [doAjax] [ajaxAction]);
} 
$usr = $mde = $act = $doAj = $ajAc = "null"; 
$usr=$1 if m|\s/([^/]+)/|; 
$mde=$1 if /m=(.+?)(&|$)/; 
$act=$1 if /a=(.+?)(&|$)/; 
$doAj=$1 if /doajax=(.+?)(&|$)/; 
$ajAc=$1 if /action=(.+?)(&|$)/; 
print join "|", ($usr,$mde,$act,$doAj,$ajAc)' file 
[user]|[module]|[action]|[doAjax]|[ajaxAction]
userx|contacts|form|null|null
usery|customer|null|request|getContacts
userx|meeting|null|date|null

A better (more readable output) approach would be to use printf instead:

2

If you prefer to do this in awk, you can do as follows. Split lets you split a string with any field separator.

awk  '{split($7,a,"/"); split(a[3],b,"m="); split(b[2],c,"&"); split(c[2],d,"="); print a[2], c[1], d[1], d[2] }' logfile

This generates the desired columns.

userx contacts a form
usery customer doajax request
userx meeting doajax date

The remaining step is to format. Arrays in awk are associative, and can be indexed with strings - see here. You can do the following; here, op (short for output) is initialized to null. Then, we set op[d[1]]=d[2].

awk  '{split($7,a,"/"); split(a[3],b,"m="); split(b[2],c,"&"); split(c[2],d,"="); op["a"]="null"; op["doajax"]="null"; op["ajaxaction"]="null"; op[d[1]]=d[2];print a[2], c[1], op["a"], op["doajax"], op["ajaxaction"] }' junk.txt 

[modified to]

awk  '{split($7,a,"/"); split(a[3],b,"m="); split(b[2],c,"&"); split(c[2],d,"="); op["a"]="null"; op["doajax"]="null"; op["action"]="null"; op[d[1]]=d[2]; split(c[3],f,"="); split(f[2],g,"."); op[f[1]]=g[1]; print a[2], c[1], op["a"], op["doajax"], op["action"] }' junk.txt 

Output is as follows

userx contacts form null null
usery customer null request getContacts
userx meeting null date null
  • I realize that for usery on the section of actionAjax is null when it should be getContacts How can I do for to change that? – Sara Quispe Apr 26 '17 at 21:43
  • add the following : split(c[3],f,"="); split(f[2],g,"."); op[f[1]]=g[1]; I modified the solution above – Vignesh Apr 26 '17 at 21:53
0
perl -lane '
BEGIN {
   print $H = join "|", map { s/.*/[$&]/r } @H = qw/user module action doAjax ajaxAction/;
   pos($H) = 0;
   push(@pos, pos($H)-$p), $p=pos($H) while $H =~ /\[/g;
   $fmt = join "", map { "\%-${_}s" } @pos[1..$#pos], length($H)-$p;
}

   my(%h, %H) = $F[-1] =~ /[?&]\K([^=]+)=([^&]+)/g;
   @H{@H} = ($F[-1] =~ m|^/([^/]+)|, @h{qw/m a doajax action/});
   print sprintf $fmt, map { $H{$_} // "null" } @H;
' logfile

Results

[user]|[module]|[action]|[doAjax]|[ajaxAction]
userx  contacts form     null     null
usery  customer null     request  getContacts
userx  meeting  null     date     null

Explanation

  1. Perl options:

    -l makes ORS = RS = \n

    -a stores fields in array @F obtained by splitting current record on /\s+/, so e.g., $F[0] => $1, $F[1] => $2, ..., $F[-1] => $NF

    -n sets up an implicit loop which reads the input file line by line AND no output unless asked for.

  2. BEGIN block:

    First we print the header. Then we determine the format dynamically based on the header. For every line read, we setup a hash %h whose keys are the strings before = and values are the string after =. The string to be looked at is abutted by ? or & on the left and & on the right. Next, we set up another hash %H whose keys are renamed versions of %h hash. Next we print the hash based on the format we computed in BEGIN block.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.