85

The Windows dir directory listing command has a line at the end showing the total amount of space taken up by the files listed. For example, dir *.exe shows all the .exe files in the current directory, their sizes, and the sum total of their sizes. I'd love to have similar functionality with my dir alias in bash, but I'm not sure exactly how to go about it.

Currently, I have alias dir='ls -FaGl' in my .bash_profile, showing

drwxr-x---+  24 mattdmo  4096 Mar 14 16:35 ./
drwxr-x--x. 256 root    12288 Apr  8 21:29 ../
-rw-------    1 mattdmo 13795 Apr  4 17:52 .bash_history
-rw-r--r--    1 mattdmo    18 May 10  2012 .bash_logout
-rw-r--r--    1 mattdmo   395 Dec  9 17:33 .bash_profile
-rw-r--r--    1 mattdmo   176 May 10  2012 .bash_profile~
-rw-r--r--    1 mattdmo   411 Dec  9 17:33 .bashrc
-rw-r--r--    1 mattdmo   124 May 10  2012 .bashrc~
drwx------    2 mattdmo  4096 Mar 24 20:03 bin/
drwxrwxr-x    2 mattdmo  4096 Mar 11 16:29 download/

for example. Taking the answers from this question:

dir | awk '{ total += $4 }; END { print total }'

which gives me the total, but doesn't print the directory listing itself. Is there a way to alter this into a one-liner or shell script so I can pass any ls arguments I want to dir and get a full listing plus sum total? For example, I'd like to run dir -R *.jpg *.tif to get the listing and total size of those file types in all subdirectories. Ideally, it would be great if I could get the size of each subdirectory, but this isn't essential.

  • 5
    Why does't ls -lh help you ? It prints total sum in top. You can also run du -sh *.exe to get disk space usage information in human readable form. – bagavadhar Apr 16 '13 at 19:40
  • @ashwin I don't know what the 'total' ls -lh is printing, but it's not always related to what the awk scripts below calculate, or what I add up by hand. Sometimes it's close to the number of KB of files in the directory, but it doesn't seem to take the allocated sizes of subdirectories into effect. I'd be grateful if you could point me toward an explanation of what exactly that number is... – MattDMo Apr 17 '13 at 17:05
  • see if my answer below works for you – bagavadhar Apr 17 '13 at 20:17
  • 2
    ls -lh does not show the total of size of a dir calculating it's contents – aequalsb Feb 9 '17 at 14:35
  • 1
    One liner: du -ach *.exe – tuga Jun 6 at 12:31

11 Answers 11

28

The following function does most of what you're asking for:

dir () { ls -FaGl "${@}" | awk '{ total += $4; print }; END { print total }'; }

... but it won't give you what you're asking for from dir -R *.jpg *.tif, because that's not how ls -R works. You might want to play around with the find utility for that.

| improve this answer | |
  • 1
    This is true, if you're looking for essentially the contents size of each file, NOT the size the file consumes on disk. This distinction is more pronounced for very small files. On my distro's each file is allocated space on disk in 4 KB chunks (so a 300 byte file still uses 4K on a disk, as reported by the du command). Given that's what the OP was looking for "how much space each file takes up", then du is the way to do it. – Jon V Jan 10 '17 at 20:20
  • 2
    dir is already the name of a popular GNU coreutil, I'd rather not name a function like that. – dessert Nov 3 '17 at 8:18
  • 1
    Please fix so it works on both Linux and OS-S and your "-a" include "." and "..", which is not good : ) Here's the fixed command: dir () { ls -FAl "${@}" | awk '{ total += $5; print }; END { print "total:"total }'; } – Dmitry Shevkoplyas Sep 11 '19 at 14:39
  • @DmitryShevkoplyas running Cygwin bash on Windows, ls -a reports the size of . and .. as 0 bytes, so it doesn't affect the total. However, your point is valid on Linux and OSX, so I've changed my function definition of dir accordingly. Thanks! – MattDMo Jun 6 at 17:21
183

There's already a UNIX command for this: du

Just do:

du -ach 

As per convention you can add one or more file or directory paths at the end of the command. -h is an extension to convert the size into a human-friendly format, -a gives you the "apparent" size (file size instead of disk usage), and -c gives a total at the end.

| improve this answer | |
  • 15
    Yes, du works fine. You can use the -c option (same as --total) to get a total at the end of the list. – MikeB May 21 '14 at 16:28
  • 11
    Note that du gives the disk usage, not the sum of file sizes. – Stéphane Chazelas Feb 9 '15 at 10:41
  • 8
    du -h doesn't sum the sizes of the files passed to it. du -h *.so shows the size of each file, but not the sum. I think what you're wanting here is du -hc *.so (or even du -hc *.so | tail -1). But of course, he wants the directory listing, too. – Limited Atonement Jan 15 '16 at 17:10
  • 1
    This command works only with short list of files. See what happens when you have 850000 files in a directory!!! – hamidfzm May 20 '16 at 10:07
  • 8
    ! -a means --all. Consider rather using --apparent-size – Arnauld VM Nov 9 '18 at 13:31
27

You can use du -h -c directory|tail -1

This will generate a single line with memory usage.

| improve this answer | |
  • 20
    du -hs directory if you only want the total. – Stéphane Chazelas Feb 9 '15 at 10:41
8

Simply print the current line that you are summing the total of:

dir | awk '{ print; total += $4 }; END { print "total size: ",total }'
| improve this answer | |
6

with perl:

perl -le 'map { $sum += -s } @ARGV; print $sum' -- *.pdf

Size of all non-hidden PDF files in current directory.

| improve this answer | |
1

Adding the following to .bash_profile or .bashrc works for me.

dir ()
{
find . -iname "$@" -exec ls -lh {} \;
find . -iname "$@" -print0|xargs -r0 du -csh|tail -n 1;
}

Now when i do a dir *.mp3 it does recursively and prints total at the end. You can control how much depth you want by adding a maxdepth parameter to the find. I know running find twice is not a very effiecnt idea. But i couldnt think of a better way. Atleast it gets the job done.

| improve this answer | |
1

Using du and a awk statement like the one mentioned above will provide what you are looking for.

Example: du /home/abc/Downloads/*.jpg | awk '{ print; total += $1 }; END { print "total size: ",total }'

This will list all files in folder Downloads of user abc ending in .jpg and prints the sum of all these files at the end of the listing.

| improve this answer | |
1

For counting files in a directory using a mask, I tend to follow this approach:

For bytes

du -ac --bytes  | grep "zip$" | awk '{ print; total += $1 }; END { print "total lobsters: ", total, " Bytes" }'

For Kilobytes

du -ac --bytes  | grep "zip$" | awk '{ print; total += $1 }; END { print "total lobsters: ", total/1024, " KB" }'

For Megabytes

du -ac --bytes  | grep "zip$" | awk '{ print; total += $1 }; END { print "total lobsters: " total/1024/1024 " MB" }'

You get the idea.

The breakdown is simple:

  • du - disk usage
    • -a - all files
    • -c - total bytes
    • --bytes - print output in bytes [in newer versions of bash, not sure if this applies anymore]
  • grep - global regular expression print [prints output matching patterns]
    • "zip$" - pattern to match. 'zip' is the string, and the '$' denotes end of string/line/etc - in this case, match lines that END with 'zip'. Conversely, placing '^' at the start of the string indicates that the pattern will be at the start of the string [ie: "^start" will match lines beginning with the word 'start'] - With this knowledge, wrapping a string in ^ and $ respectively, will match lines that start/end with the pattern used. "^hello people$" will match strings saying 'hello people'. "^hello(.*)people$" will match strings saying 'hello french people' and 'hello coding people', but not 'hello coding people with no lives'
  • awk - a scripting language programmed by Aho, Weinberger, and Kernighan. Not a very original name, but a very powerful language that is excellent for text processing and data extraction.
    • { print; total += $1 }
      • print - print the line currently being iterated over
      • total += $1 - initialize variable total if it isn't already, and add the first block separated by the field separator, in this case a space character. This can be changed by the -F flag.
      • ; - line / statement terminator. you can put multiple awk statements on a single line using this, similar to terminating linux command line statements. Otherwise you could have them in a multi line thing still surrounded by { ... }
      • END - this effectively means that awk will execute the actions specified before it exits.
    • { print "total lobsters: " total " Bytes" }
      • print "total lobsters: " - first part of string being output
      • total - the variable containing the total sum of lines iterated over
      • " Bytes" - final part of printed string, tacked on to the end of the prior two statements
      • obviously, these three statements are encapsulated in { } like the first part.

So, stepping through an example in a case where we want to count the total number of zip files in a directory:

du -ac --bytes

836544  ./wp-content/themes/astra.1.8.1.zip
934364  ./wp-content/themes/astra.2.0.1.zip
400033  ./wp-content/uploads/2019/09/premium-addons-for-elementor-3.2.9-WJdFQT1mLd3GA81lQEAo.zip
117351218       ./wp-content/uploads/backwpup-fc5928-temp/2019-05-30_00-47-01_TX6FSKC601.zip
1192275 ./wp-content/plugins/essential-addons-elementor-master.zip
170     ./wp-content/plugins/gravityforms/images/doctypes/icon_zip.gif
1969651 ./wp-content/plugins/acf.zip
4284    ./wp-content/plugins/types/application/controllers/api/handler/import_from_zip_file.php

Two components of output, col 1 represented by the numeric values: 836544, 934364... etc, and col 2 being the path of the file.

However, since there are two lines that do not match what we want - icon_zip.gif and import_from_zip_file.php - we want to exclude these. Since du does not provide a way to filter recursively by extension (that I know of), we filter using grep

grep "zip$"

This effectively has the output from du piped to it, and filters the lines that end in zip, eliminating the two records we don't want:

836544  ./wp-content/themes/astra.1.8.1.zip
934364  ./wp-content/themes/astra.2.0.1.zip
400033  ./wp-content/uploads/2019/09/premium-addons-for-elementor-3.2.9-WJdFQT1mLd3GA81lQEAo.zip
117351218       ./wp-content/uploads/backwpup-fc5928-temp/2019-05-30_00-47-01_TX6FSKC601.zip
1192275 ./wp-content/plugins/essential-addons-elementor-master.zip
1969651 ./wp-content/plugins/acf.zip

Then awk parses each line, with the numerics in col 1 being stored in $1

We get this:

836544  ./wp-content/themes/astra.1.8.1.zip
934364  ./wp-content/themes/astra.2.0.1.zip
400033  ./wp-content/uploads/2019/09/premium-addons-for-elementor-3.2.9-WJdFQT1mLd3GA81lQEAo.zip
117351218       ./wp-content/uploads/backwpup-fc5928-temp/2019-05-30_00-47-01_TX6FSKC601.zip
1192275 ./wp-content/plugins/essential-addons-elementor-master.zip
1969651 ./wp-content/plugins/acf.zip
total lobsters:  128.339  MB
| improve this answer | |
0

To get both, dir output and size calculation, without using any of the other proposed options, you can use tee(1) and process substitution...

dir | tee >( awk '{ total += $4 }; END { print total }' )
| improve this answer | |
  • No need for tee, just dir | awk '{print; total += $4}; END {print total}' – Stéphane Chazelas Feb 9 '15 at 13:27
0
du path_to_your_files/*.jpg | awk '{ total += $1 }; END { print total }'
| improve this answer | |
  • 3
    No. First, just giving a command is not an answer. Second, if you'd bothered to read the whole question, the other answers, and my comments, you'd have seen this is NOT what I want. – MattDMo Aug 26 '15 at 18:36
  • To be fair to the poster, this is the first search result for the Google search linux sum human readable sizes and is what I'm looking for. – Sridhar Sarnobat Aug 18 '19 at 7:25
-2
du * | awk -v sum=0 '{print sum+=$1}' | tail -1
| improve this answer | |
  • This doesn't do what I was asking for. – MattDMo Jun 3 '15 at 21:24

Not the answer you're looking for? Browse other questions tagged or ask your own question.