2

I have a tab-delimited text file, that is around 3K lines long.

I want to calculate frequency of data appearing in it, in 10,000 ranges.

Input.txt

Price  
500    
1500   
10001   
15000  
17000  
25000  
33000  
39000  
:
:
8000000 

Output.txt

Pricerange      pricerangecount  
0-10000          2               
10000-20000      3               
20000-30000      1               
30000-40000      2 
:
:  
7
awk 'BEGIN{print "Pricerange\tpricerangecount"}
     NR>1 {cur=int($0/10000); A[cur]+=1; if (cur>m) m=cur; }
     END {for(i=0;i<=m;i++) printf("%d-%d:\t%d\n",i*10000, (i+1)*10000, A[i])}' 
  • Small cosmetical change suggested: awk 'BEGIN{print "Pricerange\tpricerangecount"} {cur=int($0/10000); A[cur]+=1; if (cur>m) m=cur; } END {for(i=0;i<=m;i++) printf("%d-%d:\t%d\n",i*10000, (i+1)*10000, A[i])}' – tink Oct 18 '12 at 21:02
  • @tink, right and we should skip first line in input too. But then no longer comfortably a one-liner. :-) – dubiousjim Oct 18 '12 at 21:05
4

I generally use gsl-histogram from the GNU Scientific Library package. As one-liner as it gets in your case (without pretty-printing, the previous solutions are all right, and I have upvoted them):

  tail -n+2 Input.txt | gsl-histogram 0 40000 4
  • 1
    Nice. To skip the header it should be tail -n+2. – Thor Oct 19 '12 at 15:54
3

Perl:

perl -ne'$h{int$_/10000}++;END{printf"%d0000-%d0000  %d\n",$_,$_+1,$h{$_}for sort{$a<=>$b}keys%h}'

expanded:

while(<>) { #read each line (-n)
  $h{int $_/10000}++; #count buckets
}
for (sort {$a<=>$b} keys %h) { #sort numerically
  printf "%d0000-%d0000  %d\n", $_, $_+1, $h{$_};
}
2

Here's a coreutils and dc alternative:

<Input.txt tail -n +2 | while read; do <<< "$REPLY 10000 /p" | dc; done | sort -n | uniq -c \
   | while read cnt rng; do 
       printf "%-15s %-15s\n" \
         $(<<< "$rng 10000 *p" | dc)-$(<<< "$rng 1 + 10000 *p" | dc) $cnt
     done > Output.txt

Output.txt:

0-10000         2
10000-20000     3
20000-30000     1
30000-40000     2
  • there is no need to sort ; here a solution with bash shopt -s extglob; N=10000; while read n;do [[ $n = *(0-9) ]]&&((a[n/N]++));done; for i in ${!a[*]};do echo $((i*N))-$(((i+1)*N-1)) $((a[i]));done – Nahuel Fouilleul Oct 19 '12 at 16:08
  • 1
    uniq is counting occurrences so sorting is needed. You should post your suggestion as an answer. – Thor Oct 20 '12 at 0:57
1

Although bash is not a good solution, following a comment, here another solution

N=10000
while read n;do [[ $n =~ ^[0-9]*$ ]]&&((a[n/N]++));done
for i in ${!a[*]};do echo $((i*N))-$(((i+1)*N-1)) $((a[i]));done

it will be faster on small files because there isn't the overhead of spawning a new process but less efficient on large files.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.