To find the average time taken to create certain files, I'm using this minutes array, then would simply use bash arithmetic to find the average. However, I'm unable to get the difference except for the first pair of elements. Here l is the array of subtractions of the i++ and i; what's wrong?

MMarray=(`ls -lrt /some/location/ |tail -57|head -55|tr -s " "|cut -d" " -f8|cut -c4,5`)
arrLen=`echo ${#MMarray[@]}`

    echo ${l[$i]}

Also, how would 02-59 subtraction be handled?

  • Leading zero are interpreted as octal in bash. So while 02-59 is OK. 08 - 59 would crash. (And e.g. 020 would be decimal 16.) To specify base one can say base#number, e.g. (( 10#08 )) or (( 10#$some_var )). – Runium Jan 30 '15 at 10:11
  • 1
    Also note that you can say (( x = i + 1 )) instead of x=$(($i + 1)). – Runium Jan 30 '15 at 10:15
  • 1
    Also note that you can say arrLen=${#MMarray[@]}. (No need for echo.) – Runium Jan 30 '15 at 10:20
  • Which is the sole reason I had to put the #0 at the end of the variables j and k, thanks though! What Im unable to comprehend is how come the length of array l is being 1, as I've already put the ( ), need to print all the differences, could see the differences are being calculated if put set -ex in script, unable to store/print :( – Keyshov Borate Jan 30 '15 at 11:02
  • 1
    To append to the array say ((l[i] = j - k)) or l[i]=$(( j - k )) or l+=( $(( j - k )) ) – for the latest you have to declare the array before the loop, as in: l=(), or declare -a l=(). The declare specifically set it as a non-associative array. If you use declare -A the array becomes associative - but then the keys are also in random order, i.e. if you say for k in .... mywiki.wooledge.org/BashGuide/Arrays – Runium Jan 30 '15 at 11:24

If you know times are not much over one hour, you can simply add 60 if the result is negative. However, this is not the way to do it anyway. There are several points I'd like to raise:

  1. NEVER parse output of ls, especially not the time part. It depends on the locale and can give completely unpredictable results.
  2. Why assume times are around a hour and just like at the minutes, if you can just subtract dates? Use the unix timestamp - the number of second since the beginning of the world (1970). That's the standard way of storing time.
  3. Just write let i++ and let l=j-k, don't use millions of parentheses
  4. You are making $l an array of length 1, and then access $i th member. This is why it only works for the first pair.
  5. You are using very c-like programming style. Use a loop of the form for i in "${MMarray[@]}"; do... and just save the previous one.
  6. You are using too much arrays and loops, do everything in a stream of dates, it will be much better. So pretty much everything can be rewritten.
  7. You do realize, that an average time is simply first minus last divided by the number of files minus 1 (the number of diffences)? You don't need to subtract and then add together again...

What to do if you want to get all the time differences in seconds, one per line:

find /some/location -type f -name '*your filter which files you want*' -printf '%T@\n' | sort -n | awk 'NR>1 { print  $1-previous } {previous = $1}'

What it does? find finds the files and prints the unix timestamps. If you don't specify -name it lists eveything. -type f means only files, no directories. sort -n sorts the times numerically. awk simply computes and prints differences.

For average, just keep the first and the last date. For instance, for all the files in the current directory:

find . -type f -printf '%T@\n' | sort -n | awk 'NR==1 { first = $1 } END{ print ($1-first)/(NR-1) }'

The results are decimal numbers in seconds, but you can easily just write int(($1-first)/(NR-1)) for an integer or int(($1-first)/(NR-1))/60 for minutes.

  • Would be grateful! Array's range is [0-60], also negative subtractions are rare, as for the #5 It works, aint it? The data I've is just HH:MM of these files last modified timestamps in ls -lrt so how would literally subtracting dates would help is unclear, please enlighten, also, is it possible to convert all HH:MM to epoch seconds, that would help a lot as then it just would need / & % with sixties. – Keyshov Borate Jan 30 '15 at 8:38
  • 1
    Don't convert anything by hand, just give --time-style +%s argument to ls to get epoch, if you really want to do it this way. If you really only have HH:MM then just do let minutes+=hours*60 and after that do the subtraction. If you get negative numbers, just do if [ $difference -lt 0 ]; then let difference+=60; fi (or 1440 if you already added the hours*60). Still... average difference is last minus first divided by the number of differences, so all this is unnecessary. – orion Jan 30 '15 at 8:44
  • There were actually three types of these .prop files, which would be generating randomly, however older run's files will still be there in same location, so calculating average time for all of these three kinds of files would be little difficult as ls -lrt kind-name.prop would be having older files too which can't be considered. – Keyshov Borate Jan 30 '15 at 8:48
  • 1
    That's why you give a glob pattern, something like -iname myfile*.prop or whatever. Or you can still squeeze tail and head between find and sort like you did before. And if you really want, you can still use ls and all that cutting and translating and pasting instead of find and keep the awk script. Also, I really recommend epoch times and sorting - who says the files were really generated alphabetically? – orion Jan 30 '15 at 8:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.