23

This is the command I am using to list some files:

find . -name \*.extract.sys -size +1000000c -exec ls -lrt {} \;
-rw-r--r--   1 qa1wrk15   test       1265190 Sep 29 01:14 ./var/can/projs/ar/rep/extract/Sep/29/ar.ARAB-PI_7.20110929.extract.sys
-rw-r--r--   1 qa1wrk15   test       1345554 Sep 29 01:14 ./var/can/projs/ar/rep/extract/Sep/29/ar.ARAB-PI_2.20110929.extract.sys
-rw-r--r--   1 qa1wrk15   test       1370532 Sep 29 01:14 ./var/can/projs/ar/rep/extract/Sep/29/ar.ARAB-PI_3.20110929.extract.sys
-rw-r--r--   1 qa1wrk15   test       1399854 Sep 29 01:14 ./var/can/projs/ar/rep/extract/Sep/29/ar.ARAB-PI_8.20110929.extract.sys

and so on.

Now I want to calculate the total size of these files by summing up the 5th column. I thought of using awk, to do this so I tested the following in a particular directory

>ls -lrt | awk `{ print $1 }`
ksh: syntax error at line 1 : `{' unmatched

I don't understand what is the problem, why this syntax error.

I am thinking to try

ls -lrt | awk `BEGIN {total = 0} {for(i=0;i<NR;i++){total+=$5}} END {printf "%d",total} 

this also, but a simple awk script is not working.

Please suggest or correct me if I am wrong, or if there is a workaround for this.

0

5 Answers 5

29

First of all, you should use straight single quotes ('), not the inclined ones (`).

The awk inline script could be as follow:

ls -lrt | awk '{ total += $5 }; END { print total }'

so, no need to initialize total (awk initializes it to zero), and no need to loop, awk already executes the script on every line of input.

3
  • 1
    In summary, the one-liner would be this: find . -name \*.extract.sys -size +1000000c -exec ls -lrt {} \; | awk '{ total += $5 }; END { print total }'
    – helt
    Commented Apr 27, 2016 at 15:35
  • 1
    Don't parse ls!
    – OrangeDog
    Commented Jul 10, 2018 at 16:29
  • @enzotib : there are some rows where third column is blank, so the sum of 5th column is not working right, is there anyway to accommodate those
    – Learner
    Commented Mar 12, 2020 at 17:20
16

@enzotib has already pointed out what your syntax error is - I'm going to go off on a little tangent.

Summing a column of numbers is one of those things that keeps popping up. I've ended up with this shell function:

sumcol() 
{ 
    awk "{sum+=\$$1} END {print sum}"
}

With this, your solution becomes:

ls -lrt | sumcol 5

That will sum the numbers in column 5 and print the value.

2
  • In effect, I was wondering why there is not a standard utility for that.
    – enzotib
    Commented Sep 29, 2011 at 9:34
  • Goes right into my .aliases file. Commented Sep 29, 2011 at 12:04
16

Here is another way to do this by using du:

find . -name \*.extract.sys -size +1000000c -print0 | du -c --files0-from=- | awk 'END{print $1}'
3
  • 1
    Excellent use of du. Nice example. As an added benefit, you can add the "-h" option to du in order to get the output in Gig. Then you don't have to count digits and such. I just ran a test on a bunch of files that are approximately 1188800 kbytes in size. How big is that? No matter, du -ch --files0-from=- tells me that my total is "9.0G". Sweeeet!
    – Mike S
    Commented Nov 9, 2016 at 18:21
  • Sweet, excellent use of du!
    – haridsv
    Commented Jan 25, 2017 at 4:53
  • Also, if you are only interested in the total, then pipe du output to tail -1.
    – haridsv
    Commented Jan 25, 2017 at 5:00
5

In case HP-UX find also has -printf option, that would be useful to compose a formula for bc to calculate:

( find . -name \*.extract.sys -size +1000000c -printf '%s+'; echo 0 ) | bc

If the formula gets too big, it can be calculated step-by-step:

( find . -name \*.extract.sys -size +1000000c -printf 's+=%s\n'; echo s ) | bc
3

With GNU find:

find -name '*.extract.sys' -size +1000000c -printf '%s\n' | jq -s add

With GNU stat:

find . -name '*.extract.sys' -size +1000000c -exec stat -c%s '{}' + | jq -s add

With BSD stat:

find . -name '*.extract.sys' -size +1000000c -exec stat -f%z '{}' + | jq -s add
1
  • Is jq a utility? Don't seem to find on debian distribution (Debian 4.9.51-1 (2017-09-28) x86_64 GNU/Linux) Commented Sep 19, 2019 at 19:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .