Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I'm looking for an existing Linux command-line tools that can accomplish the following:

Consider there is an input stream /dev/inputstream which spews an infinite stream of data that looks like this:

A
A
B
A
C
Z
A
.
.
.

NOTE: (Each line is one character from the alphabet).

The tool is used in the following fashion:

cat /dev/inputstream | tool

The output of the tool should be like the one from "top" command. It should be a continuously updated list of top N frequently occurring items.

Eg:

A -- 10 times
B -- 8 times
Z -- 7 times
C -- 2 times
D -- 1 time

If the stream were not infinite I could have accomplished this using sort and uniq.

share|improve this question
    
I think you will need to write a script to do what you want. I don't believe that there is a utility that will do this exactly like you want. –  datUser May 15 at 20:34
    
@datUser I thought so too until I found the tool logtop which I put in an answer below. –  Prashanth Ellina May 15 at 21:16
    
Wow nice find, logtop looks pretty useful too. –  datUser May 16 at 12:47

2 Answers 2

up vote 6 down vote accepted

I found this tool that matches my need exactly, called logtop. According to the documentation:

logtop displays real-time count of strings received in standard input.

Examples

Is useful for some cases, like getting the IP flooding your server:

$ tail -f /var/log/apache2/access.log | cut -d' ' -f1 | logtop

Or the top buzzing article of your blog:

$ tail -f /var/log/apache2/access.log | cut -d' ' -f7 | grep article | logtop

Fedora/CentOS/RHEL

There is no RPM that I could find for this application but it's pretty trivial to download, compile and install manually. It does require 2 dependencies to be installed to be able to compile it.

$ sudo yum install uthash ncurses-devel

You can download the software from Github:

$ git clone https://github.com/JulienPalard/logtop.git

To build it:

$ cd logtop
$ make

You can then install it or run it from this directory, ./logtop.

Example

$ tr -dc 'A-Z' </dev/urandom | fold -w 1 | head -n 10MB | ./logtop
10000 lines, 10000.00 lines/s
RANK    CNT   LINE/S LINE                                                                                                                             
   1    428   428.00 T
   2    417   417.00 G
   3    416   416.00 S
   4    414   414.00 K
   5    413   413.00 Q
   6    412   412.00 E
   7    401   401.00 I
share|improve this answer
    
I was able to install it effortlessly using apt-get install logtop on Ubuntu 14.04. –  Prashanth Ellina May 15 at 21:18

In Perl:

perl -nle '
          $N = 5;      # Change as needed
          $freq{$_}++;
          if ( keys %freq >= $N ){
              @sorted = sort { $freq{$b} <=> $freq{$a} } keys %freq;
              for ( 0..$N-1){
                  $key = $sorted[$_];
                  print "$key -- $freq{$key} times";
              }
          }
          sleep 5;
          system(clear)
          ' < /dev/inputstream
share|improve this answer
    
That sleeps for 5 seconds for every line of input, so won't probably be very useful. –  Stéphane Chazelas May 15 at 19:03
    
@StephaneChazelas fair enough but I think there m7st be some waiting intwrbal ptherwise there won't be any time to read the display. –  Joseph R. May 15 at 20:20

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.