I have to parse huge text files where certain lines are of interest and others are not. Within those of interest I have to count the occurrences of a certain keyword.

Assumed the file is called input.txt and it looks like this:


I want to exclude the lines with ignore and count the number of format300, how do I do that?

What I've got so far is this command which only counts ONCE PER LINE (which is not yet good enough):

cat input.txt | grep -v ignore | grep 'format300' | wc -l

Any suggestions? If possible I want to avoid using perl.

4 Answers 4


This one-liner should be able to do what you want:

grep -v ignore input.txt | sed 's/format300/format300\n/g' | grep -c "format300"

basically you are replacing each occurrence of your keyword with the keyword itself and a newline character, which effectively makes your input stream have the keyword only once on any given line. Then grep -c is counting lines with your keyword in them.

  • 3
    I would replace the sed with tr ',' '\n' otherwise you're going to count format3000s as well not just format300s
    – iruvar
    Apr 8, 2016 at 15:18
  • 2
    @1_CR, agreed, but you would also have to use grep -xc format300 instead of grep -c format300 to avoid false positives on "format3000". So full solution is grep -v ignore input.txt | tr , '\n' | grep -xc format300
    – Wildcard
    Apr 8, 2016 at 18:57
  • @Wildcard, indeed
    – iruvar
    Apr 8, 2016 at 19:01
  • Thanks guys, voted for this answer, but there was a simpler answer which I preferred over this one.
    – basZero
    Apr 11, 2016 at 10:50

You don't need the first cat, that it is known as a Useless use of cat (UUOC).

Also, very useful is grep -o, that only outputs the matching patterns, one per line.

And then, count lines with wc -l.

grep -v ignore YOUR_FILE | grep -o format300 | wc -l

This prints 3 for your small sample.

  • But it should print out 5, because my sample contains 5 times format300
    – basZero
    Apr 11, 2016 at 7:46
  • But there are 2 format300 on the line containing ignore, so these 2 shouldn't count, right? Apr 11, 2016 at 7:59
  • thanks, this is the perfect answer to my question, very simple and it does exactly what I asked for.
    – basZero
    Apr 11, 2016 at 10:49
  • and yes, you are right of course regarding the ignore. All perfect!
    – basZero
    Apr 11, 2016 at 10:51

Input file may potentially contain partial matches that would invalidate the result, for example:

1 format300,format250,format300
2 format250,ignore,format160,format300,format300
3 format250,format250,format300
4 format999,format300000,format999
5 format999,ignore_me_not,format300

You don't want to count format300000 on line 4 or ignore line 5 because ignore_me_not contains the substring ignore.

This would do the trick:

grep -v "\bignore\b" FILE |grep -o "\bformat300\b"|wc -l

Correct output is


..becase line 2 is ignored, line 5 is not, and line 4 doesn't contain exactly format300.

If you let the wc -l part out, you can see what exactly is being matched:

enter image description here

  • Clever, but it goes beyond his actual stated requirements: "I want to exclude the lines with ignore..." Your point about format3000 et. al. is perfectly valid, though. (As I commented on another answer.)
    – Wildcard
    Apr 8, 2016 at 19:00
  • This is the only solution that is considering the false positives, the others so far will all fail with this input. There is still a problem, though, because the word boundaries aren't quite what are needed. A character like - is not a word character, so ignore-me-not will match \bignore\b but shouldn't. Apr 8, 2016 at 19:06

A Perl way:

perl -lne '$k+=(s/format300//g) unless /ignore/; }{ print $k' input.txt 

The s/format300//g will replace all occurrences of format300 with nothing, and returns the number of replacements. It's a simple way of counting the occurrences. The number is then added to $k and the whole thing only happens if the line doesn't match ignore. The }{ is perl shorthand for "do this after you've finished reading the file, so print $k will print the total number found.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.