I have a logfile which name/directory is: /srv/uni/kani/LogDay.09-03-2016 The name of the logfile changes date every day. So tomorrow will be: `LogDay.10-03-2016' and so on...

I would like help to create a script who looks through that file and finds this line:

08-03-2016 19:00:01 b5121 DISPLAY ID(IP[RANDOMIPADRESS] PID[22003]) MSG[BEGIN JOB /srv/uni/PROD/PNRUN/PN4949D

and measures the time in minutes until this job is written in the log:

08-03-2016 19:19:17 b7285 DISPLAY ID(IP[RANDOMIPADRESS] JOB[FO1008] PID[28085]) MSG[END JOB /srv/uni/PROD/PNRUN/FO1008D, , 27392]

There is alot of other lines in the log.. with other jobs saying BEGIN job and END job and so on. But its those 2 specific i want to measure the time between.

I would like to use this script at the end of the evening to see how long all the jobs took totally. Instead of having to go in the log and count manually...

And I want to put it in a bash script :)

I would like the output to say something like. Tonights work took totally X hour X minutes.

Can anyone help me with that?

  • 2
    Are there corresponding time stamps with those log messages? How else do we know the elapsed time??
    – Jeff Schaller
    Commented Mar 9, 2016 at 15:18
  • Do you mean in real-time, or after the fact? i.e. do you mean watch the log for MSG[BEGIN JOB /srv/uni/PROD/PNRUN/PN4949D to arrive and then count seconds until MSG[END JOB /srv/uni/PROD/PNRUN/FO1008D is written, or do you mean look at old logs and work it out? Please post an extract from the log file showing us exactly what's in it and clarify if you mean real-time, or post-process. edit your original question, don't add it to the comments! Commented Mar 9, 2016 at 15:35
  • Yes! The logfile contains timestamps! I would like to run this script at the end of the night to see how long everything took. Looks like this: 08-03-2016 19:00:01 b5121 DISPLAY ID(IP[RANDOMIPADRESS] PID[22003]) MSG[BEGIN JOB /srv/uni/PROD/PNRUN/PN4949D and the other line looks like 08-03-2016 19:19:17 b7285 DISPLAY ID(IP[RANDOMIPADRESS] JOB[FO1008] PID[28085]) MSG[END JOB /srv/uni/PROD/PNRUN/FO1008D, , 27392]
    – David
    Commented Mar 9, 2016 at 15:51
  • Please edit that output into your Q; the formatting makes a comment unusable.
    – Jeff Schaller
    Commented Mar 9, 2016 at 17:02
  • Ive now updated the Q !
    – David
    Commented Mar 9, 2016 at 17:27

1 Answer 1

    myDate=$(date +'%d-%m-%Y')
    BEGINJOB="PN4949D" # Can set to $1 to accept as 1st arg
    ENDJOB="FO1008D" # Can set to $2 to accept as 2nd arg
    regexFirst="$myDate.*BEGIN JOB /srv/uni/PROD/PNRUN/$BEGINJOB"
    regexLast="$myDate.*END JOB /srv/uni/PROD/PNRUN/$ENDJOB"
    # Find First time 
    FIRST=$(sed -n "/$regexFirst/p" $logPathLogDay.$myDate  | head -1)
    FIRST=$(echo $FIRST | sed -n 's/.*\([0-2][0-9]:[0-5][0-9]:[0-5][0-9]\).*/\1/p')
    echo "First job started at $FIRST"
    # Find Last time
    LAST=$(sed -n "/$regexLast/p" $logPathLogDay.$myDate | tail -1)
    LAST=$(echo $LAST | sed -n 's/.*\([0-2][0-9]:[0-5][0-9]:[0-5][0-9]\).*/\1/p')
    echo "Last job ended at $LAST"
    #Compare time
    # feeding variables by using read and splitting with IFS
    IFS=: read first_hour first_min first_sec <<< "$FIRST"
    IFS=: read last_hour last_min last_sec <<< "$LAST"
    # the 10# is there to avoid errors with leading zeros
    # by telling bash that we use base 10
    total_hours=$((10#$last_hour - 10#$first_hour))
    total_minutes=$((10#$last_min - 10#$first_min))
    total_secs=$((10#$last_sec - 10#$first_sec))
    echo "Tonight's work took $total_hours hours $total_minutes minutes and $total_secs seconds"

This script will look at the current date, and will search for that dates file, and the first occurrence of BEGIN and the last occurrence of END and then compare the time stamps.

Output like:

First job started at 19:00:01

Last job ended at 24:00:01

Tonight's work took 5 hours 0 minutes and 0 seconds

This is a quick script I threw together, you can clean it and modify output, throw it into a function if needed etc.

*EDIT: Also, to run this script just add to your crontab to run daily at a time you prefer.

**EDIT: I added the variables for the begin and end regex.

I turned what looks to be the Job Reference # as variables, so you can easily change it or even have it passed as an argument.

Lastly, I put the path to the log as a variable so that can be changed easily as well.

As a side note, I added the comments in the script, albeit not the most detailed, but they are there so you can follow the script and make changes to suit your needs. As with any script, the customization can be endless!

You can also modify the date to be like: myDate=$(date --date="1 day ago" +'%d-%m-%Y') if you prefer to have this look through previous day logs.

  • That looks great! But now it looks for any BEGIN and any END. i want it to look for the start of: BEGIN JOB /srv/uni/PROD/PNRUN/PN4949D and the END job of END JOB /srv/uni/PROD/PNRUN/FO1008D. And i would like to be able to run it from any directory even though the log is in /srv/uni/kani/ catalog
    – David
    Commented Mar 10, 2016 at 6:04
  • I had updated the answer to be more strict with the regex query, also it can be run from any directory as the path is absolute.
    – J L
    Commented Apr 20, 2016 at 15:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .