0

I want to extract time from all of my .log files and add all of these times into a single entity. My log files are placed in nested directories.

Directory structure for my log files:

|-- temp-system
|   |-- bash
|   |   |-- bash-4.3-branch_update-5.patch
|   |   |-- build.log
|   |   |-- build.sh
|   |   `-- DONE
|   |-- binutils
|   |   |-- build.log
|   |   |-- build.sh
|   |   `-- DONE
|   |-- build-variables
|   |   |-- build.log
|   |   |-- build.sh
|   |   `-- DONE
|   |-- bzip2
|   |   |-- build.log
|   |   |-- build.sh
|   |   `-- DONE
|   |-- check
|   |   |-- build.log
|   |   |-- build.sh
|   |   `-- DONE
|   |-- cloog
|   |   |-- build.log
|   |   |-- build.sh
|   |   `-- DONE
|   |-- gettext
|   |   |-- build.log
|   |   |-- build.sh
|   |   `-- DONE
|-- build-system
|   |-- gcc
|   |   |-- build.log
|   |   |...

Snippet of my log files:

are/man/man1'
make[2]: Leaving directory `/tmp/panda64/temp-system/texinfo/texinfo-5.2/man'
make[1]: Leaving directory `/tmp/panda64/temp-system/texinfo/texinfo-5.2/man'
make[1]: Entering directory `/tmp/panda64/temp-system/texinfo/texinfo-5.2'
make[2]: Entering directory `/tmp/panda64/temp-system/texinfo/texinfo-5.2'
make[2]: Nothing to be done for `install-exec-am'.
make[2]: Nothing to be done for `install-data-am'.
make[2]: Leaving directory `/tmp/panda64/temp-system/texinfo/texinfo-5.2'
make[1]: Leaving directory `/tmp/panda64/temp-system/texinfo/texinfo-5.2'
/tmp/panda64/temp-system/texinfo

real    1m59.973s
user    1m26.352s
sys     0m11.820s

Every log file has these last three lines real, user and sys. I want to extract all these out of my log files and add them and output in a format HH:MM:SS. I was trying to achieve this using cat and grep but haven't been able to get the regex right.

UPDATE

To traverse the directory, I'm using the following code:

list=(temp-system build-system)
for direcs in ${list[@]}; do
    case $direcs in
        temp-system )
            pushd $direcs
                _list1=(bash binutils build-variables)
                for subdirecs in ${_list1[@]}; do
                    case $subdirecs in
                        * )
                            # execute code to add the time
                            # and later add this with other
                            # time calculated from other directories
                            ;;
                    esac
                done
            popd;;
    esac
done
  • I don't know the output format of time good enough but grep -r --include *.log '\(real\|user\|sys\)\s\+[0-9]\+' temp-system should give you all lines containing real, user or sys plus some spacing plus something beginning with a number. – jschlichtholz May 28 '16 at 7:58
  • 'add them' - do you mean a+b calculation or just putting to the end of a single sile? For calculation you definitely will need longer way to go. – Putnik Jun 3 '16 at 6:24
  • @Putnik I want to do a+b calculation – Abhimanyu Saharan Jun 3 '16 at 10:32
1

Instead of that huge script to traverse the files you can just use find:

find /var/log -name *.log -exec ls {} \;

Replace the path and the 'ls' in the -exec to fit your needs.

As for grepping for the time you can do something like:

grep "^real\t" time.txt | awk '{ print  $NF }'

I cannot come up with a pretty solution to formatting it though.

  • And how do I add the times? Is there an easy way to do it or will I have to get my hands dirty? – Abhimanyu Saharan May 29 '16 at 22:05
  • I think you'll have to get your hands dirty since the output is minutes then seconds. I cannot find a nice way to do it since the output could look like 1m4.006s. I was toying around with something like grep "^real\t" time.txt | awk '{ print $NF }' | sed 's/m/:/' | sed 's/.....$//' but then you would easily miss some 0's, ie. the above would become 1:4. – Mikael Kjær May 30 '16 at 2:27
0

Once you have a file containing times in the form 1m59.973s (eg. the output from Mikael Kjær's answer), you can transform them into a single sum for bc(1):

( cat alltimes.log | sed 's_h_*3600+_;s_m_*60+_;s_s_+_' | tr -d '\n' ; echo 0 ) | bc

That prints a number of seconds (including decimal places). Then bash (whose arithmetic expansion doesn't handle decimals) can break it up and reformat it:

TOTAL=`( cat alltimes.log | sed 's_h_*3600+_;s_m_*60+_;s_s_+_' | tr -d '\n' ; echo 0 ) | bc`
INT=${TOTAL%.*}
DEC=${TOTAL#*.}
H=$((INT/3600))
M=$((INT/60%60))
S=$((INT%60))
printf %02i:%02i:%02i.%s $H $M $S $DEC

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.