I want to measure the time between two jobs in a logfile

Question

I have a logfile which name/directory is: /srv/uni/kani/LogDay.09-03-2016 The name of the logfile changes date every day. So tomorrow will be: `LogDay.10-03-2016' and so on...

I would like help to create a script who looks through that file and finds this line:

08-03-2016 19:00:01 b5121 DISPLAY ID(IP[RANDOMIPADRESS] PID[22003]) MSG[BEGIN JOB /srv/uni/PROD/PNRUN/PN4949D

and measures the time in minutes until this job is written in the log:

08-03-2016 19:19:17 b7285 DISPLAY ID(IP[RANDOMIPADRESS] JOB[FO1008] PID[28085]) MSG[END JOB /srv/uni/PROD/PNRUN/FO1008D, , 27392]

There is alot of other lines in the log.. with other jobs saying BEGIN job and END job and so on. But its those 2 specific i want to measure the time between.

I would like to use this script at the end of the evening to see how long all the jobs took totally. Instead of having to go in the log and count manually...

And I want to put it in a bash script :)

I would like the output to say something like. Tonights work took totally X hour X minutes.

Can anyone help me with that?

Answer 1

 #!/bin/bash
    
    myDate=$(date +'%d-%m-%Y')
    logPath='/srv/uni/kani/catalog/'
    BEGINJOB="PN4949D" # Can set to $1 to accept as 1st arg
    ENDJOB="FO1008D" # Can set to $2 to accept as 2nd arg
    regexFirst="$myDate.*BEGIN JOB /srv/uni/PROD/PNRUN/$BEGINJOB"
    regexLast="$myDate.*END JOB /srv/uni/PROD/PNRUN/$ENDJOB"
    # Find First time 
    FIRST=$(sed -n "/$regexFirst/p" $logPathLogDay.$myDate  | head -1)
    FIRST=$(echo $FIRST | sed -n 's/.*\([0-2][0-9]:[0-5][0-9]:[0-5][0-9]\).*/\1/p')
    echo "First job started at $FIRST"
    
    # Find Last time
    LAST=$(sed -n "/$regexLast/p" $logPathLogDay.$myDate | tail -1)
    LAST=$(echo $LAST | sed -n 's/.*\([0-2][0-9]:[0-5][0-9]:[0-5][0-9]\).*/\1/p')
    echo "Last job ended at $LAST"
    #Compare time
    # feeding variables by using read and splitting with IFS
    IFS=: read first_hour first_min first_sec <<< "$FIRST"
    IFS=: read last_hour last_min last_sec <<< "$LAST"
    
    # 
    # the 10# is there to avoid errors with leading zeros
    # by telling bash that we use base 10
    total_hours=$((10#$last_hour - 10#$first_hour))
    total_minutes=$((10#$last_min - 10#$first_min))
    total_secs=$((10#$last_sec - 10#$first_sec))
    
    echo "Tonight's work took $total_hours hours $total_minutes minutes and $total_secs seconds"

This script will look at the current date, and will search for that dates file, and the first occurrence of BEGIN and the last occurrence of END and then compare the time stamps.

Output like:

First job started at 19:00:01

Last job ended at 24:00:01

Tonight's work took 5 hours 0 minutes and 0 seconds

This is a quick script I threw together, you can clean it and modify output, throw it into a function if needed etc.

*EDIT: Also, to run this script just add to your crontab to run daily at a time you prefer.

**EDIT: I added the variables for the begin and end regex.

I turned what looks to be the Job Reference # as variables, so you can easily change it or even have it passed as an argument.

Lastly, I put the path to the log as a variable so that can be changed easily as well.

As a side note, I added the comments in the script, albeit not the most detailed, but they are there so you can follow the script and make changes to suit your needs. As with any script, the customization can be endless!

You can also modify the date to be like: myDate=$(date --date="1 day ago" +'%d-%m-%Y') if you prefer to have this look through previous day logs.