Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have a tab-delimited text file, that is around 3K lines long.

I want to calculate frequency of data appearing in it, in 10,000 ranges.

Input.txt

Price  
500    
1500   
10001   
15000  
17000  
25000  
33000  
39000  
:
:
8000000 

Output.txt

Pricerange      pricerangecount  
0-10000          2               
10000-20000      3               
20000-30000      1               
30000-40000      2 
:
:  
share|improve this question
add comment

5 Answers 5

up vote 7 down vote accepted
awk 'BEGIN{print "Pricerange\tpricerangecount"}
     NR>1 {cur=int($0/10000); A[cur]+=1; if (cur>m) m=cur; }
     END {for(i=0;i<=m;i++) printf("%d-%d:\t%d\n",i*10000, (i+1)*10000, A[i])}' 
share|improve this answer
    
awesome oneliner! –  jack Oct 18 '12 at 20:42
    
Small cosmetical change suggested: awk 'BEGIN{print "Pricerange\tpricerangecount"} {cur=int($0/10000); A[cur]+=1; if (cur>m) m=cur; } END {for(i=0;i<=m;i++) printf("%d-%d:\t%d\n",i*10000, (i+1)*10000, A[i])}' –  tink Oct 18 '12 at 21:02
    
@tink, right and we should skip first line in input too. But then no longer comfortably a one-liner. :-) –  dubiousjim Oct 18 '12 at 21:05
add comment

Perl:

perl -ne'$h{int$_/10000}++;END{printf"%d0000-%d0000  %d\n",$_,$_+1,$h{$_}for sort{$a<=>$b}keys%h}'

expanded:

while(<>) { #read each line (-n)
  $h{int $_/10000}++; #count buckets
}
for (sort {$a<=>$b} keys %h) { #sort numerically
  printf "%d0000-%d0000  %d\n", $_, $_+1, $h{$_};
}
share|improve this answer
add comment

I generally use gsl-histogram from the GNU Scientific Library package. As one-liner as it gets in your case (without pretty-printing, the previous solutions are all right, and I have upvoted them):

  tail -n+2 Input.txt | gsl-histogram 0 40000 4
share|improve this answer
1  
Nice. To skip the header it should be tail -n+2. –  Thor Oct 19 '12 at 15:54
    
@Thor: Thanks a bunch, corrected! –  Deer Hunter Oct 19 '12 at 16:03
add comment

Here's a coreutils and dc alternative:

<Input.txt tail -n +2 | while read; do <<< "$REPLY 10000 /p" | dc; done | sort -n | uniq -c \
   | while read cnt rng; do 
       printf "%-15s %-15s\n" \
         $(<<< "$rng 10000 *p" | dc)-$(<<< "$rng 1 + 10000 *p" | dc) $cnt
     done > Output.txt

Output.txt:

0-10000         2
10000-20000     3
20000-30000     1
30000-40000     2
share|improve this answer
    
there is no need to sort ; here a solution with bash shopt -s extglob; N=10000; while read n;do [[ $n = *(0-9) ]]&&((a[n/N]++));done; for i in ${!a[*]};do echo $((i*N))-$(((i+1)*N-1)) $((a[i]));done –  Nahuel Fouilleul Oct 19 '12 at 16:08
1  
uniq is counting occurrences so sorting is needed. You should post your suggestion as an answer. –  Thor Oct 20 '12 at 0:57
add comment

Although bash is not a good solution, following a comment, here another solution

N=10000
while read n;do [[ $n =~ ^[0-9]*$ ]]&&((a[n/N]++));done
for i in ${!a[*]};do echo $((i*N))-$(((i+1)*N-1)) $((a[i]));done

it will be faster on small files because there isn't the overhead of spawning a new process but less efficient on large files.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.