How to count the lines containing one of two words but not both

Question

I need to count the lines containing the words the and an in a text file (poem.txt), but not those containing both.

I've tried using

grep -c the poem.txt | grep -c an poem.txt

but this gives me the wrong answer of 6 when the total number of the and an is 9 lines.

I do want to count the lines containing the words and not the words themselves. Only the actual word should count, so the but not there, and an but not Pan.

Example file: poem.txt

Where is the misty shark?
Where is she?
The small reef roughly fights the mast.
Where is the small gull?
Where is he?
The gull grows like a clear pirate.
Clouds fall like old mainlands.

She will Rise calmly like a dead pirate.
Eat an orange.
Warm, sunny sharks quietly pull a cold, old breeze.
All ships command rough, rainy sails.

Elvis Aaron Presley also known simply as the Elvis
He is also referred to as the King
The best-selling solo music artist of all time
He was the most commercially successful artist in many genres

He has many awards including a Grammy lifetime achievement
Elvis in the 1970s has numerous jumpsuits including an eagle one.

To clarify further: how many lines in the poem contain the or an but you should not count the lines that include both the and an.

the car is red - this counted
an apple is in the corner - not counted
hello i am big - not counted
where is an apple - counted

So here the output should be 2.

Edit: I'm not worried about case sensitivity.

Final edit: Thanks for all your help. i've managed to solve the problem. i used the one of the answer and changed it a little. i used cat poem.txt | grep -Evi -e '\<an .* the\>' -e '\<the .* an\>' | grep -Eci -e '\<(an|the)\> how ever i did change the -c in the second grep to a -n for some added info. Yet again thank you for all the help!! :)

Answer 1

With grep:

cat poem.txt \
  | grep -Evi -e '\<an\>.*\<the\>' -e '\<the\>.*\<an\>' \
  | grep -Eci -e '\<(an|the)\>'

This counts the matched lines. You can find an alternative syntax which counts the total number of matches down below.

Breakdown:

The frist grep command filters out all lines containing both 'an' and 'the'. The second grep command counts those lines, containing either 'an' or 'the'.

If you remove the c from the second grep's -Eci, you will see all matches highlighted.

Details:

• The -E option enables extended expression syntax (ERE) for grep.

• The -i option tells grep to match case-insensitive

• The -v option tells grep to invert the result (i.e. match lines not containing the pattern)

• The -c option tells grep to output the number of matched lines instead of the lines themselves

• The patterns:

  1. \< matches the beginning of a word (thanks @glenn-jackman)
  2. \> matches the end of a word (thanks @glenn-jackman)

  --> That way we can make sure to not match words containing 'the' or 'an' (like 'pan')

  3. grep -Evi -e '\<an\>.*\<the\>' thus matches all lines not containing 'an ... the'

  4. Similarly, grep -Evi -e '\<the\>.*\<an\>' matches all lines not containing 'the ... an'

  5. grep -Evi -e '\<an\>.*\<the\>' -e '\<the.*an\>' is the combination of the 3. and 4.

  6. grep -Eci -e '\<(an|the)\>' matches all lines containing either 'an' or 'the' (surrounded by whitespace or start/end of line) and prints the number of matched lines

EDIT 1: Use \< and \> instead of ( |^) and ( |$), as suggested by @glenn-jackman

EDIT 2: In order to count the number of matches instead of the number of matched lines, use the following expression:

cat poem.txt \
  | grep -Evi -e '\<an\>.*\<the\>' -e '\<the\>.*\<an\>' \
  | grep -Eio -e '\<(an|the)\>' \
  | wc -l

This uses the -o option of grep, which prints every match in a separate line (and nothing else) and then wc -l to count the lines.

Answer 2

perl -nE 'END {say $c+0} ++$c if /\bthe\b/i xor /\ban\b/i' file

gawk 'END {print c+0} /\<the\>/ != /\<an\>/ {++c}' IGNORECASE=1 file

Comparing the results from matching each expression can give the outcome you want.

For example, the result of matching \<the\> may be either 0 or 1. If the result of the other match is the same, then both regexps were either found or not found, and the line should not be counted. If they differ it means that one match was found and the other was not, so the counter is incremented.

gawk has a built-in xor() function:

gawk 'END {print c+0} xor(/\<the\>/,/\<an\>/) {++c}' IGNORECASE=1 file

Answer 3

The following GNU awk program should do the trick:

awk '(/(^|\W)[Tt]he(\W|$)/ && !/(^|\W)[Aa]n(\W|$)/) || (/(^|\W)[Aa]n(\W|$)/ && !/(^|\W)[Tt]he(\W|$)/) {c++} END{print c}' poem.txt

This will increase the counter c, if either

• the line matches (^|\W)[Tt]he(\W|$) (first-letter-case-insensitive the, preceded by non-word constituent (\W) or begin of line (^), and followed by non-word constituent (\W) or end-of line ($)) but not (^|\W)[Aa]n(\W|$) (the isolated first-letter-case-insensitive an) - OR -
• the line matches (^|\W)[Aa]n(\W|$) but not (^|\W)[Tt]he(\W|$)

In the end, print the value of c.

It can be formulated slightly shorter using \< and \> for "beginning-of-word" and "end-of-word":

awk '(/\<[Tt]he\>/ && !/\<[Aa]n\>/) || (/\<[Aa]n\>/ && !/\<[Tt]he\>/) {c++} END{print c}' poem.txt

Even shorter would be:

awk '/\<[Tt]he\>/ != /\<[Aa]n\>/ {c++} END{print c}' poem.txt

as the inequality is only ever true if either, but not both (nor none) of an and the are present on a line.

This approach requires GNU awk because the \W and \< / \> constructs are GNU extensions to the extended regular expression syntax (but \< / \> are also understood by BSD regexes).

Notice that the pipeline construct you showed in your own attempted solution won't work, as calling grep with a file as input parameter supersedes reading from stdin, so the first part of the pipeline would simply vanish unnoticed, with the output being entirely due to the last part (which looks for occurences of an, even those embedded in other words).

Answer 4

You can do it with GNU grep and PCRE zero-length assertions:

grep -iP '(?=.*\bthe\b)(?!.*\ban\b)|(?=.*\ban\b)(?!.*\bthe\b)' poem.txt

Where is the misty shark?
...
Eat an orange.
...

grep -ciP '(?=.*\bthe\b)(?!.*\ban\b)|(?=.*\ban\b)(?!.*\bthe\b)' poem.txt

9

The same feature is available in perl (where it originated), and perl may be present on machines where GNU grep isn't.

Answer 5

This will let you output lines containing any number of any set of words and will work using any awk in any shell on every Unix box:

$ cat tst.awk
BEGIN {
    FS = "[^[:alnum:]_]+"
    split(tolower(tgts),tmp)
    for (i in tmp) {
        targets[tmp[i]]
    }
    tgtsRequired = (tgtsRequired ? tgtsRequired : 1)
}
{
    delete present
    for (i=1; i<=NF; i++) {
        present[tolower($i)]
    }

    tgtsPresent = 0
    for (target in targets) {
        tgtsPresent += (target in present ? 1 : 0)
    }
}
tgtsPresent == tgtsRequired

$ awk -v tgts='the an' -f tst.awk poem.txt
Where is the misty shark?
The small reef roughly fights the mast.
Where is the small gull?
The gull grows like a clear pirate.
Eat an orange.
Elvis Aaron Presley also known simply as the Elvis
He is also referred to as the King
The best-selling solo music artist of all time
He was the most commercially successful artist in many genres

$ awk -v tgts='the an' -v tgtsRequired=2 -f tst.awk poem.txt
Elvis in the 1970s has numerous jumpsuits including an eagle one.

$ awk -v tgts='the an eagle' -v tgtsRequired=2 -f tst.awk poem.txt

$ awk -v tgts='the an eagle' -v tgtsRequired=3 -f tst.awk poem.txt
Elvis in the 1970s has numerous jumpsuits including an eagle one.

It assumes a "word" is any contiguous string of alpha-numeric-or-underscore characters.

Answer 6

This GNU awk command will do what you need:

$ awk '
        function lnxor(a,b) {return !(a&&b||(!a&&!b))};
        { str=tolower($0); a=(str~/\<the\>/); b=(str~/\<an\>/); 
          c=lnxor(a,b); count+=c 
        } END {print count}
' poem.txt

9

The lnxor is the logical not xor of a and b.

Where a is true if the line contains the and b is true if the line contains an.

In this way it is quite easy to extend the idea to more words.

In GNU awk it is also possible to use IGNORECASE (set it to non-zero), instead of tolower(), to ignore the case of the string being matched by the regex. However, that will change all string and regex operations, including the effects of FS and RS. Being, as it is, such an extensive change (which will work in this case), it should be used with more care. It is not specific to some particular test.

Answer 7

A method using perl wherein we sum the boolean values of the regexes and using that as an index select from an anonymous array the appropriate increment depending on whether none found, any one found, or both.

perl -lne '$k += (0,1,0)[/\ban\b/i+/\bthe\b/i]}{print 0+$k' poems.txt
9

Answer 8

Using gawk and wc:

gawk 'xor(/\<an\>/,/\<the\>/)' IGNORECASE=1 ./poem.txt | wc -l