20

This is the command I am using to list some files:

find . -name \*.extract.sys -size +1000000c -exec ls -lrt {} \;
-rw-r--r--   1 qa1wrk15   test       1265190 Sep 29 01:14 ./var/can/projs/ar/rep/extract/Sep/29/ar.ARAB-PI_7.20110929.extract.sys
-rw-r--r--   1 qa1wrk15   test       1345554 Sep 29 01:14 ./var/can/projs/ar/rep/extract/Sep/29/ar.ARAB-PI_2.20110929.extract.sys
-rw-r--r--   1 qa1wrk15   test       1370532 Sep 29 01:14 ./var/can/projs/ar/rep/extract/Sep/29/ar.ARAB-PI_3.20110929.extract.sys
-rw-r--r--   1 qa1wrk15   test       1399854 Sep 29 01:14 ./var/can/projs/ar/rep/extract/Sep/29/ar.ARAB-PI_8.20110929.extract.sys

and so on.

Now I want to calculate the total size of these files by summing up the 5th column. I thought of using awk, to do this so I tested the following in a particular directory

>ls -lrt | awk `{ print $1 }`
ksh: syntax error at line 1 : `{' unmatched

I don't understand what is the problem, why this syntax error.

I am thinking to try

ls -lrt | awk `BEGIN {total = 0} {for(i=0;i<NR;i++){total+=$5}} END {printf "%d",total} 

this also, but a simple awk script is not working.

Please suggest or correct me if I am wrong, or if there is a workaround for this.

26

First of all, you should use straight single quotes ('), not the inclined ones (`).

The awk inline script could be as follow:

ls -lrt | awk '{ total += $5 }; END { print total }'

so, no need to initialize total (awk initializes it to zero), and no need to loop, awk already executes the script on every line of input.

  • 1
    In summary, the one-liner would be this: find . -name \*.extract.sys -size +1000000c -exec ls -lrt {} \; | awk '{ total += $5 }; END { print total }' – helt Apr 27 '16 at 15:35
  • Don't parse ls! – OrangeDog Jul 10 '18 at 16:29
16

@enzotib has already pointed out what your syntax error is - I'm going to go off on a little tangent.

Summing a column of numbers is one of those things that keeps popping up. I've ended up with this shell function:

sumcol() 
{ 
    awk "{sum+=\$$1} END {print sum}"
}

With this, your solution becomes:

ls -lrt | sumcol 5

That will sum the numbers in column 5 and print the value.

  • In effect, I was wondering why there is not a standard utility for that. – enzotib Sep 29 '11 at 9:34
  • Goes right into my .aliases file. – Konrad Rudolph Sep 29 '11 at 12:04
14

Here is another way to do this by using du:

find . -name \*.extract.sys -size +1000000c -print0 | du -c --files0-from=- | awk 'END{print $1}'
  • 1
    Excellent use of du. Nice example. As an added benefit, you can add the "-h" option to du in order to get the output in Gig. Then you don't have to count digits and such. I just ran a test on a bunch of files that are approximately 1188800 kbytes in size. How big is that? No matter, du -ch --files0-from=- tells me that my total is "9.0G". Sweeeet! – Mike S Nov 9 '16 at 18:21
  • Sweet, excellent use of du! – haridsv Jan 25 '17 at 4:53
  • Also, if you are only interested in the total, then pipe du output to tail -1. – haridsv Jan 25 '17 at 5:00
4

In case HP-UX find also has -printf option, that would be useful to compose a formula for bc to calculate:

( find . -name \*.extract.sys -size +1000000c -printf '%s+'; echo 0 ) | bc

If the formula gets too big, it can be calculated step-by-step:

( find . -name \*.extract.sys -size +1000000c -printf 's+=%s\n'; echo s ) | bc
1

With GNU find:

find -name '*.extract.sys' -size +1000000c -printf '%s\n' | jq -s add

With GNU stat:

find . -name '*.extract.sys' -size +1000000c -exec stat -c%s '{}' + | jq -s add

With BSD stat:

find . -name '*.extract.sys' -size +1000000c -exec stat -f%z '{}' + | jq -s add
  • Is jq a utility? Don't seem to find on debian distribution (Debian 4.9.51-1 (2017-09-28) x86_64 GNU/Linux) – Death Metal Sep 19 '19 at 19:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.