2

I currently have the following command

wget -O - -q http_url_goes_here |
    grep -Eci '(cart|shop|buy|order|store|shipping|products|price)'

It gives me a count of ALL occurences of each of the following keywords on the site (cart|shop|buy|order|store|shipping|products|price).

The problem I have is that I get counts as high as 200 if some of the words occur multiple times. I would like to get a count of between 0 and 8, which would indicate how many of the keywords were found on the site. As in the count should be incremented by a maximum of 1 for each keyword.

I appreciate your help.

4

With GNU grep:

wget -O - -q http_url_goes_here |
    grep -wEoi 'cart|shop|buy|order|store|shipping|products|price' | 
        sort -fu | wc -l

I added the -w to your grep command so that you don't count, for example, bishop as a hit for shop.

  • Thank you very much. And thanks to everyone else as well.. – rho Sep 9 '14 at 16:29
2

POSIXly:

tr -cs '[:alnum:]_' '[\n*]' | grep -xiF 'cart
shop
buy
order
store
shipping
products
price' | sort -fu | wc -l

You can also have the list of words in a file and use:

tr -cs '[:alnum:]_' '[\n*]' | grep -xiFf word.list | sort -fu | wc -l

You can possibly make it more efficient by doing:

tr -cs '[:alnum:]_' '[\n*]' | awk -v o=0 '
   {$0=tolower($0)}
   NR==FNR{a[$0];n++;next}
   {
     for (i in a) if ($0 == i) {
       if (++o == n) exit
       delete a[i]
     }
   }
   END {print o}' word.list -
  • So cool! A question, why do we need _? – cuonglm Sep 9 '14 at 15:30
  • @Gnouc, _ is generally considered a word component (like by your -w), so that cart and cart_2 are two different words. – Stéphane Chazelas Sep 9 '14 at 15:37
  • And in awk solution, I think {$0=tolower($0)} should be after `NR==FNR{...} since when we don't need lowercase word in word.list, it has been lowercase already. – cuonglm Sep 9 '14 at 15:44
  • @Gnouc, well we want both to be in lower case if we're to compare them. – Stéphane Chazelas Sep 9 '14 at 15:46
0

The answer of Volker Siegel, is very useful. However if you want the output to be a number between 0 and 8 you should add a grep to count the lines.

$ wget -O - -q ix.de | grep -Eoi '(cart|shop|buy|order|store|shipping|products|price)' | sort -i | uniq -i | grep -c ''
2

Note that now the flag -c of uniq is not necessary.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.