# frequency of occurence

I have a tab-delimited text file, that is around 3K lines long.

I want to calculate frequency of data appearing in it, in 10,000 ranges.

Input.txt

``````Price
500
1500
10001
15000
17000
25000
33000
39000
:
:
8000000
``````

Output.txt

``````Pricerange      pricerangecount
0-10000          2
10000-20000      3
20000-30000      1
30000-40000      2
:
:
``````

``````awk 'BEGIN{print "Pricerange\tpricerangecount"}
NR>1 {cur=int(\$0/10000); A[cur]+=1; if (cur>m) m=cur; }
END {for(i=0;i<=m;i++) printf("%d-%d:\t%d\n",i*10000, (i+1)*10000, A[i])}'
``````
• Small cosmetical change suggested: awk 'BEGIN{print "Pricerange\tpricerangecount"} {cur=int(\$0/10000); A[cur]+=1; if (cur>m) m=cur; } END {for(i=0;i<=m;i++) printf("%d-%d:\t%d\n",i*10000, (i+1)*10000, A[i])}' – tink Oct 18 '12 at 21:02
• @tink, right and we should skip first line in input too. But then no longer comfortably a one-liner. :-) – dubiousjim Oct 18 '12 at 21:05

I generally use `gsl-histogram` from the GNU Scientific Library package. As one-liner as it gets in your case (without pretty-printing, the previous solutions are all right, and I have upvoted them):

``````  tail -n+2 Input.txt | gsl-histogram 0 40000 4
``````
• Nice. To skip the header it should be `tail -n+2`. – Thor Oct 19 '12 at 15:54

Perl:

``````perl -ne'\$h{int\$_/10000}++;END{printf"%d0000-%d0000  %d\n",\$_,\$_+1,\$h{\$_}for sort{\$a<=>\$b}keys%h}'
``````

expanded:

``````while(<>) { #read each line (-n)
\$h{int \$_/10000}++; #count buckets
}
for (sort {\$a<=>\$b} keys %h) { #sort numerically
printf "%d0000-%d0000  %d\n", \$_, \$_+1, \$h{\$_};
}
``````

Here's a coreutils and dc alternative:

``````<Input.txt tail -n +2 | while read; do <<< "\$REPLY 10000 /p" | dc; done | sort -n | uniq -c \
| while read cnt rng; do
printf "%-15s %-15s\n" \
\$(<<< "\$rng 10000 *p" | dc)-\$(<<< "\$rng 1 + 10000 *p" | dc) \$cnt
done > Output.txt
``````

Output.txt:

``````0-10000         2
10000-20000     3
20000-30000     1
30000-40000     2
``````
• there is no need to sort ; here a solution with bash `shopt -s extglob; N=10000; while read n;do [[ \$n = *(0-9) ]]&&((a[n/N]++));done; for i in \${!a[*]};do echo \$((i*N))-\$(((i+1)*N-1)) \$((a[i]));done` – Nahuel Fouilleul Oct 19 '12 at 16:08
• `uniq` is counting occurrences so sorting is needed. You should post your suggestion as an answer. – Thor Oct 20 '12 at 0:57

Although bash is not a good solution, following a comment, here another solution

``````N=10000
while read n;do [[ \$n =~ ^[0-9]*\$ ]]&&((a[n/N]++));done
for i in \${!a[*]};do echo \$((i*N))-\$(((i+1)*N-1)) \$((a[i]));done
``````

it will be faster on small files because there isn't the overhead of spawning a new process but less efficient on large files.