0

Heys guys I wanted to count the number of words that end with 's' from this URL,

https://matt.might.net/articles/what-cs-majors-should-know/

Here's what I did

curl https://matt.might.net/articles/what-cs-majors-should-know/ | \
grep s$ 

I couldn't find a way to filter out the words that end with 's' and count them

2
  • 2
    How do you define a word. Is <canvas> a word ending in >? Do you want to find the words in the HTML source, or in some rendition of the HTML into text? If the latter, how should that rendition be done, etc. May 9, 2022 at 14:16
  • So like after curling the URL I get the HTML of a text file and I want to count the total number the words that end with 's'. May 9, 2022 at 14:18

1 Answer 1

3
elinks -no-numbering -no-references -dump https://matt.might.net/articles/what-cs-majors-should-know/ |
  grep -Po '\w+s\b' |
  wc -l

(with GNU grep or compatible)

Gives me: 595

elinks retrieves the HTML and converts it to text in its own way. Some alternatives are w3m -dump or lynx -nolist.

Then we look for the sequences of one or more word characters in there (word characters being alnums or underscores) followed by a s provided it's not followed by a word character, and count then with wc -l.

In essence, the definition of a word there is a sequence of 2 or more alnum or underscore characters and we look for the ones ending in s (add a -i option to grep to also look for those ending in S). That means the s in it's for instance is not counted. If you want wanted to consider single-letter words, you could replace + with * above. Or just do grep -Po 's\b' but then you wouldn't be able to see the list of words when omitting the | wc -l.

Another approach would be to extract the words first and find the ones ending in s:

grep -Eo '\w+' | grep -c 's$'

In any case, that's quite a crude definition of word that only works properly for simple English text. When applied to text in other languages, you could run into problems. For instance, it would find a match inside the abrogowałybyście Polish word (picked at random in a word list) if that ś was encoded in its decomposed sU+0301 form (where U+0301 is the combining acute accent) as that s would be seen as not being followed by a word character.

You may also want to consider what to do about the S in USA or U.S.A..

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .