14

I have a directory with a lot of photos in it. Specifically, du -sh --apparent-size /path/to/myfolder gives me 331G. Which is great. But now I want to get a listing grouped by month, e.g. something like this:

2016-01   20MB
2016-02  520MB
2016-03  312MB
...

Is there a (reasonable) way to do this with linux builtins, or should I just write my own Python utility to do it?

3
  • 1
    Linux doesn't have builtins, it's an operating system kernel. Do you mean with the commands found by default in some Linux based operating system (like Debian, Fedora, ChromeOS...) instead? – Stéphane Chazelas Oct 18 '16 at 18:08
  • 8
    The Linux kernel is the linux kernel, and if I meant linux kernel builtins I would've said that. If you must be pedantic, I mean the general set of tools that you're statistically likely to have installed with a default installation of any of the top 5 linux distros. – Wayne Werner Oct 18 '16 at 20:08
  • 1
    @WayneWerner In other words, you mean GNU/Linux, including Bash, Coreutils, and other core components of the GNU operating environment. #rmswasright – Damian Yerrick Oct 19 '16 at 1:35
27

On linux, try:

find /my/path -maxdepth 1 -type f -printf '%TY-%Tm %s\n' | awk '{b[$1]+=$2} END{for (date in b) print date, b[date]}' | sort

How it works

  • find /my/path

    This looks for files in in /my/path.

  • -maxdepth 1

    This tells find not to look into subdirectories. (If you want a recursive search, then omit this option.)

  • -type f

    This tells find to limit the search to regular files.

  • -printf '%TY-%Tm %s\n'

    This tells find to print out year-month followed by the size in bytes for each file.

    Since we have no use for them, the names of the files found are not printed.

  • b[$1]+=$2

    For each file found, we add its byte count, found from column 2, to the count that that year-month combination in associative array b.

  • END{for (date in b) print date, b[date]}

    After we have processed all the output from find, we print out the results.

  • sort

    This sorts the results in date order.

Multiple line version

For those who prefer their code spread out over multiple lines:

find /my/path -maxdepth 1 -type f -printf '%TY-%Tm %s\n' |
  awk '
    {
      b[$1]+=$2
    }

    END{
      for (date in b)
        print date, b[date]
    }
    ' | sort

Example

Let's consider a directory with these files:

$ ls -l
total 27816
-rw------- 1 john1024 john1024 2459173 Nov 23  2015 img100.jpg
-rw------- 1 john1024 john1024 3479750 Nov 23  2015 img101.jpg
-rw------- 1 john1024 john1024 4028939 Nov 23  2015 img102.jpg
-rw------- 1 john1024 john1024 2928519 Jul 30 18:55 img103.jpg
-rw------- 1 john1024 john1024 2948294 Jul 30 18:55 img104.jpg
-rw------- 1 john1024 john1024 3177583 Aug  1 16:56 img105.jpg
-rw-rw---- 1 john1024 john1024 3111737 Apr 18  2016 img106.jpg
-rw-rw---- 1 john1024 john1024 1441310 Apr 18  2016 img107.jpg
-rw-rw---- 1 john1024 john1024 2430158 Apr 25 16:26 img108.jpg
-rw-rw---- 1 john1024 john1024 2424504 Apr 25 16:26 img109.jpg

The output from our command is:

$ find . -maxdepth 1 -type f -printf '%TY-%Tm %s\n' | awk '{b[$1]+=$2} END{for (date in b) print date, b[date]}' | sort
2015-11 9967862
2016-04 9407709
2016-07 5876813
2016-08 3177583

Refinements

If we want the output in mebibytes (MiB) instead of bytes, we can convert the units like this:

$ find . -maxdepth 1 -type f -printf '%TY-%Tm %s\n' | awk '{b[$1]+=$2} END{for (date in b) print date, b[date]/1024**2, "MiB"}' | sort
2015-11 9.50609 MiB
2016-04 8.97189 MiB
2016-07 5.60457 MiB
2016-08 3.03038 MiB

We can get still more control over output format by using printf. Here, to keep just one digit after the decimal point, we format the size with %5.1f:

$ find . -maxdepth 1 -type f -printf '%TY-%Tm %s\n' | awk '{b[$1]+=$2} END{for (date in b) printf "%s %5.1f MiB\n", date, b[date]/1024**2}' | sort
2015-11   9.5 MiB
2016-04   9.0 MiB
2016-07   5.6 MiB
2016-08   3.0 MiB
6
  • This is fantastic. Can you recommend any awk tutorials? I've not yet found one that hasn't made my eyes cross within about twenty seconds. – hBy2Py Oct 18 '16 at 19:36
  • 1
    @hBy2Py My favorite introduction to awk, even though it is now a bit dated, is the Grymoire tutorial. – John1024 Oct 18 '16 at 19:41
  • i suggest using printf "%s %9d\n", date, b[date] instead of print date, b[date] to add space padding to second column – rav_kr Oct 19 '16 at 9:06
  • @rav_kr Good idea. I just updated the answer with an example that uses printf. – John1024 Oct 19 '16 at 19:10
  • FWIW if you have find that supports -maxdepth you probably have [g]awk that supports PROC_INFO["sorted_in"]="@ind_str_asc" – dave_thompson_085 Oct 20 '16 at 9:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.