1

I would like to find n most common words in a text file but with the following conditions:

  • cat to get the file.
  • grep . to get the words.
  • tr -d '/r' to get rid of new lines.

I've managed to get the latter half of the code working with |sort | uniq -c | sort -nr| head -10.

  • You might want to use the -o option for grep: grep -Eo '[[:alpha:]]+ file' will print the words one per line. – glenn jackman May 15 at 23:08
1

You mention the use of grep, but I don't really see the need for it if the goal is to find the most frequent words. More likely, you just want to split the text file into words, then run your | sort | uniq -c | sort -nr | head -10 pipeline.

The solution that immediately comes to mind (there are without a doubt a number of alternative solutions) is to just use regular expressions via sed:

sed 's/\s/\n/g'

This just states to replace all white space characters (\s) with a single newline (\n). Coupled together, I would think your solution would look something like so:

cat [file] | sed 's/\s/\n/g' | sort | uniq -c | sort -nr | head -10 | awk '{print $2}'

I added in the final awk '{print $2}' to only show the words (not the count), but that's entirely dependent on the purpose of the script.

1

This sounds like a homework assignment.

Following all your requirements for cat, grep and tr, matching upper and lower case (HE=He=he=hE), as well as words ending in punctuation marks (he.=he=he,=he!):

for word in $(
  cat derp.txt | \
  tr '[:upper:]' '[:lower:]' | \
  tr -d '[:punct:]' | \
  tr -d '/r'
)
do
  words[$word]=$(cat derp.txt | grep -c $word)
  echo "${words[$word]} $word" # lots of duplicate output that's filtered out below.
done | sort -u | tail -n 10

grep . matches everything so is that a typo?

The tr -d '\r' is pointless in this context as the input field separator IFS for the for loop treats both whitespace and newlines as delimiters.

Jason K Lai's solution is probably the best.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.