Good Day. I have a log file that I would like to delete lines that are over 30 days old.

File Contents:

2017/04/04 15:53:22 [11487] building file list
2017/04/04 15:53:22 [11487] done
2017/04/04 15:53:22 [11487] sent 163 bytes  received 12 bytes  350.00 bytes/sec
2017/04/04 15:53:22 [11487] total size is 48640  speedup is 277.94
2017/04/04 15:53:29 [11493] building file list
2017/04/04 15:53:29 [11493] done

Attempted using awk, but it doesn't let you perform inline edits on the file. Also can't use gawk as a solution as I only have v 3.7.0 Was hoping someone can assist with sed perhaps? Looking for a one liner preferably. This is in bash.

  • one-liner in such case would look cumbersome. I would suggest bash solution – RomanPerekhrest May 5 '17 at 18:51
  • Agreed, perhaps a bash solution is what I should do. This is what I started with: awk 'BEGIN{FS=OFS=" "} {command="date -d \""$1"\" +'%s'";command |getline;close(command);print}' file Do you know a quick solution in bash? – AfroJoe May 5 '17 at 19:00
  • 2
    This can actually get too easy: grep -v "$(date '+%Y/%m/%d' -d '30 days ago')*" logfile – Valentin Bajrami May 5 '17 at 19:04
  • @val0x00ff, that would pull in lines that are 31 days old or older – iruvar May 5 '17 at 19:13
  • @iruvar I just dived into date's manual. The best option would be to hard code the string then. – Valentin Bajrami May 5 '17 at 19:29

If you can be sure that there will be at least one entry for each day, you can say:

sed -n '\|'$(date +'%Y/%m/%d' -d '30 days ago')'|,$p' log

Provided you have GNU sed. If not, you need to escape the slashes in the date.

  • That's standard and not GNU-specific – Stéphane Chazelas May 8 '17 at 15:40
  • @StéphaneChazelas Thanks for the correction. – Michael Vehrs May 9 '17 at 5:52
  • Sorry, I meant the sed part was standard. The date part is GNU-specific. I'd quote the $(...) ('\|'"$(date...)"'|,$p') so it still works in contexts where $IFS may contain / or digits (IOW, there's no reason you'd want to split+glob it here). – Stéphane Chazelas May 9 '17 at 6:15
grep -v "$(date '+%Y/%m/%d' -d '30 days ago')*" logfile 

If you have perl:

perl -MPOSIX -ni -e '
  BEGIN{$cutoff = strftime("%Y/%m/%d %T", localtime(time - 30*86400))}
  print if $_ ge $cutoff' file.log

If you have GNU date:

  rm -f -- "$file" &&
    awk -v "cutoff=$(date -d '30 days ago' '+%Y/%m/%d %T')" '
      $0 >= cutoff' > "$file"
} < "$file"

Both on several lines for clarity, but feel free to put then on one line if that's what you need (you'll need a ; between file.log and { and between "$file" and } when joining the lines of the awk variant.


Using dateadd and dategrep from dateutils:

dategrep  -i "%Y/%m/%d %T" ">=$(dateadd now -30d)" </path/to/logfile

dateadd calculates the datetime from now to 30 days in the past. Then dategrep only prints lines from stdin which contain a date string newer or equal (>=) than that. The date format of the log lines needs to be specified by -i in this case.

  • Can you please extend, clarify your answer? – Romeo Ninov May 9 '17 at 10:53
  • @RomeoNinov I've added some text, hope this helps. dategrep is basically similar like grep especially made to grep for dates and times. – rudimeier May 9 '17 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.