I tried to filter my log file by the functionality For example:

195.xx.x.x - - [13/Apr/2017:09:60:xx +0200] "POST /userx/index.php?m=contacts&xxxx...
192.xx.x.x - - [13/Apr/2017:09:45:xx +0200] "POST /userx/index.php?m=customer&xxxx...
197.xx.x.x - - [13/Apr/2017:09:10:xx +0200] "POST /userx/index.php?m=meeting&xxxx...
197.xx.x.x - - [13/Apr/2017:09:20:xx +0200] "POST /userx/index.php?m=dashboard&xxxx...

In this case my functionalities are contacts,customer,meeting,dashboard I try to ignore The welcome page that's by default. I used

awk '$7 !~ /m=dashboard/ ' log file

my question is If can I ignore more functionalities that are in a file ?

cat file:

For to have just this lines:

195.xx.x.x - - [13/Apr/2017:09:60:xx +0200] "POST /userx/index.php?m=contacts
192.xx.x.x - - [13/Apr/2017:09:45:xx +0200] "POST /userx/index.php?m=customer
  • You can use a delimiter as in: awk -F= '$NF != "dashboard"' log_file which should work – Valentin Bajrami Apr 13 '17 at 8:51
  • yest, but my questionis is If can I filter with words that are in a file x? – Sara Quispe Apr 13 '17 at 9:24
  • @SaraQuispe, can you elaborate your question and add the expected output? – RomanPerekhrest Apr 13 '17 at 9:39
  • 1
    use grep -Fvf file log if the keywords in file are unique in input file log – Sundeep Apr 13 '17 at 12:03
sed '/\//!{H;d};G;/m=\(.*\)\n.*\1/d;P;d' file log

Explanation: First read file with the filter keywords, then the logfile. Lines containing no / are interpreted to be keywords and appended to the hold space (H). Other lines get the hold space appended (G) and are deleted if the keyword after the m= is repeated in the keyword list (/m=\(.*\)\n.*\1/d). If not, it's printed without the appended hold space (P).

  • @Sara This solution will also remove lines with substrings of the given keywords, e.g. it will remove lines with m=meeting, if meetings is in the filter file. It can be adapted, if this is a problem. I hope an sed solution is okay for you. – Philippos Apr 13 '17 at 10:50
  • 1
    thanks @Philippos , it's works just if my request has this form /userx/index.php?m=contacts but no if is more long like that /userx/index.php?m=contacts&doajax=xxx&.. I don't know why? – Sara Quispe Apr 13 '17 at 12:41
  • Because it assumed the m parameter was the last one (you see the line break \n following the string match. If another parameter could follow you will have to modify the script to allow for either & or line feed by replacing \n with [&\n]: sed '/\//!{H;d};G;/m=\(.*\)[&\n].*\1/d;P;d' file log It's better to understand a script so you can adapt it to your needs. More questions? – Philippos Apr 13 '17 at 13:43

Since your question seems to make more sense now, I think you are looking for something like this:

awk -F= 'NR==FNR{l[$NF]=1; next} { if (!l[$NF]) print;}' ignore_words your_log_file


As Sundeep pointed out in the comment above, you could use grep as follow:

grep -Fvf ignore_words log_file

To see what the options -Fvf are used for, please consult man grep page.

  • thanks @val0x00ff , it's works just if my request has this form /userx/index.php?m=contacts but not if is more long like that /userx/index.php?m=contacts&doajax=xxx&.. I don't know why? – Sara Quispe Apr 13 '17 at 12:50
  • @SaraQuispe you are changing your requirements constantly. This is not the way to ask a question and expect someone knows exactly what you mean. You are not getting the result you want because the -F= will have 2 columns and doesn't know which one to match. – Valentin Bajrami Apr 13 '17 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.