I found a snippet of code online that would actually help with creating the frequency of each word in a text file however I would like someone to explain exactly how it is doing so

Especially the sed command since i am super new to bash and I need to know what are all the seperators are doing ( s/\(.*\)/\L\1/ ).

Here is the code:

cat EnglishText.txt
sed 's/\.//g;s/\(.*\)/\L\1/;s/\ /\n/g' EnglishText.txt | sort | uniq -c

I want to know what exactly comes after the sed, i do understand the uniq -c and the sort but i would like to know what is going on in the matching and so on .. i know this is somehow weird but again i am very new to this

In the same context of the question

sed  's/\([0-9]*\).*/\1/'

What would this mean ?

1 Answer 1


The sed script consists of three substitute commands. Substitute commands are of the form s/old/new/ which looks for something in the text that matches regular expression old and replaces it with new. If a g is put after the command, then this substitution is done repetitively ("globally"). The first one removes periods. The second one makes the text lower case. The third puts each word on its own line. In more detail:

  • s/\.//g

    This matches periods in the input and replaces them with nothing.

  • s/\(.*\)/\L\1/

    This matches anything in the input and replaces it with a lower-case version of the same.

  • s/\ /\n/g

    This replaces spaces with newlines. This has the effect of putting each word on a separate line.


Note that the period is removed and all words are lowercased and put on separate lines:

$ echo 'This test is this test.' | sed 's/\.//g;s/\(.*\)/\L\1/;s/\ /\n/g'

This form is suitable for sorting and counting:

$ echo 'This test is this test.' | sed 's/\.//g;s/\(.*\)/\L\1/;s/\ /\n/g' | sort | uniq -c
      1 is
      2 test
      2 this


The sed script, as written, does not do anything with other punctuation, like ?"!, or with tabs. With minor modifications to the above code, all can be handled:

$ echo 'This "test(?)" is this test!' | sed 's/[[:punct:]]//g; s/.*/\L&/; s/[[:space:]]/\n/g' | sort | uniq -c
      1 is
      2 test
      2 this

This uses the same type of substitute commands as the original with just small changes:

  1. s/[[:punct:]]//g removes all punctuation characters.

  2. s/.*/\L&/ converts all upper case characters to lower case.

  3. s/[[:space:]]/\n/g replaces all whitespace with newline characters.


If a line starts with a number, sed 's/\([0-9]*\).*/\1/' keeps that number and removes everything after it. All other lines are just removed. For example:

$ echo '123 tests' | sed  's/\([0-9]*\).*/\1/'
$ echo 'There are 123 tests' | sed  's/\([0-9]*\).*/\1/'
  • IF i were to include them how can i add them ?
    – JavaFreak
    Jun 20, 2016 at 2:18
  • and this is without any doubt the most beautiful explanation i read all day !
    – JavaFreak
    Jun 20, 2016 at 2:19
  • See updated answer for a suggested code that handles those issues.
    – John1024
    Jun 20, 2016 at 2:36
  • I did, can u explain these parts since the syntax i feel changed .. sorry .
    – JavaFreak
    Jun 20, 2016 at 2:38
  • sed doesn't seem to handle character classes, otherwise that would be the best way to handle e.g. punctuation. tr does, and I really wonder why this isn't a simple tr transliteration.
    – Kusalananda
    Jun 20, 2016 at 6:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .