1

I am new to unix. I am trying to find the latest log of multiple jobs and delete the latest one.

Can I write a shell script for it? I have tried with basic commands, but there it is selecting a time range and can delete the log, but it may also delete useful logs .


Sharing what I found:

touch -t 201903281325.00 start           
touch -t 201903281331.00 stop                 
find . -newer start \! -newer stop -type f \( -name "**" -o -name "**" \) -exec rm -f {} \;

This deletes all the files in that time limit


What I want to do :

Let the names of jobs are A,B,C, etc, and these run 5 times a day -- say A_1, A_2, A_3, B_1, B_2, B_3, etc.

Say today's run logs generated

Order_created_20190611_1.log
Order_created_20190611_2.log 
Order_created_20190611_3.log
Order_zip_rec_20190611_1.log
Order_zip_rec_20190611_2.log

What I have to delete automatically:

Order_created_20190611_3.log
Order_zip_rec_20190611_2.log

I want to automatically find Order_created_20190611_3.log and Order_zip_rec_20190611_2.log and delete them.

Example of job names :

  • Order_created -- job name
  • 20190611 - Date of execution
  • _2 ( mentioned at last) - is the number of time it has run. _2 means it is running 2nd time for today's run.
  • Hi roaima - have made the changes suggested bu you . please let me know if anything else is needed – Unix bug Jun 11 at 8:52
  • "AIX" -- i used uname -srm to get this . is there anything else you are asking ? – Unix bug Jun 11 at 9:28
  • That's good so far, thank you... it means the solution will be more complicated because AIX doesn't have the extensions to tools that, say, a Linux system would have. – roaima Jun 11 at 9:35
  • ok . yes. that why i hustled have not found anything concrete,thanks . please do share if you have any idea or any scripts . thanks – Unix bug Jun 11 at 9:38
  • the phrase "delete the latest" could be misleading, as it means the most recent ; it seems to me that you want to delete all of the job logs except the latest / most recent? Or rather, you want to delete the job log with the highest run number? – Jeff Schaller Jun 13 at 17:55
0

For simple filenames such as these, where you can guarantee that they will not contain strange characters such as newline, it is safe and convenient to parse the output of ls. (In the general case this is not recommended.)

As described in your question let's assume we're working with files matching these pattern, where the * will of course match "anything":

Order_created_20190611_*.log
Order_zip_rec_20190611_*.log

Additionally, for a files that is based on today's date you can find the "most recent" like this:

# Today's date in YYYYMMDD format
today="$(date +%Y%m%d)"

# Get "ls" to give us the files ordered by date modified (newest first). Pick off the first
newest="$(ls -t "Order_created_${today}_"*".log" | head -n1)"

# Show what we would delete (remove "echo" to action)
echo rm "$newest"

Using this principle you can roll this up into a loop across multiple filenames:

today="$(date +%Y%m%d)"
for prefix in Order_created Order_zip_rec
do
    newest="$(ls -t "${prefix}_${today}_"*".log" 2>/dev/null | head -n1)"
    [[ -f "$newest" ]] && echo rm "$newest"
done

Things to note

  1. You shouldn't generally assume you can parse the output of ls. I have done so in this case because your file names are well defined (and there are no GNU extensions for sort or find)
  2. I have discarded error output from ls in the case that there are no matching files
  3. Similarly, I attempt the rm only if there is a file to delete
  • Thanks i ll check this, – Unix bug Jun 11 at 10:38
  • how can i go for say if something can be before prefix and it also get searched. -- i tried this : ""_"${prefix}_${today}_"".log" 2>/dev/null in place of "${prefix}_${today}_"*".log" 2>/dev/null but it is not working for me – Unix bug Jun 11 at 13:28
0

To delete the "last" (highest-numbered "run") log file for each set of jobs and dates, I would suggest a ksh93-based solution that can keep track of the highest job runs seen for each combination as well as a list of files to delete:

#!/bin/ksh93

typeset -A highestjobruns=()
typeset -A deletions=()

for logfile in *_????????_*.log
do
  base=${logfile%.log}
  run=${base##*_}
  jobdate=${base%_?}
  if [ -z "${highestjobruns[$jobdate]}" ]
  then
        # no job runs found yet, this is it; nothing to delete yet
        highestjobruns[$jobdate]=$run
  else
        # if this file has a higher run number, it's the new target for deletion
        if [ "$run" -gt "${highestjobruns[$jobdate]}" ]
        then
                highestjobruns[$jobdate]=$run
                deletions[$jobdate]=$logfile
        fi
  fi
done
printf 'Would rm: %s\n' "${deletions[@]}"
#rm -- "${deletions[@]}"

The basic idea is to loop over every potential log file (adjust the wildcard to taste; I' requiring "anything", underscore, six digits, underscore, "anything", ending in .log), pick apart the run number and the combination of job and date, then ask some questions:

  1. Have we recorded any runs for this combination of job and date? If not, record this one and continue on. This means we won't delete a singleton job run.
  2. If we have seen previous runs of this combination of job and date, is the run we're looking at higher than the highest one we've seen so far? If so, record the new run number and populate the current file as one we might delete. Subsequent logs of the same job and date may overwrite this entry, so we'll only every delete one log file per job name and date.

At the end, I print sample rm commands for the files that we would delete; comment that out if you like, and uncomment the actual rm command itself to do the removals.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.