1

Here's an imaginary command line where "pushtolog" is a utility that accepts lines of text from a pipe and logs them into a file with a filename incorporating today's date:

tail -F mylatestlogfile.log | grep "ERROR" | pushtolog -d /tmp/myerrors/ -f yyyy-MM-dd -r errors -e log

When running on the 17th Feb 2017 the output would be logged to file:

/tmp/myerrors/errors.2017-02-17.log

At midnight on 18th Feb 2017, this file would be closed and any further output would be logged to file:

/tmp/myerrors/errors.2017-02-18.log

The switchover would happen automatically. Performance is fairly irrelevant due to a low volume of relevant log lines, so in the worst case it would be fine for the utility to check the date every time it wrote a log line.

The GNU/Linux server that this will run on has a minimal set of utilities installed. It has Python 2 installed, and I could write a utility myself. Before taking that step, it's sensible to ask whether there is a tool that will already the job, or whether there's an idiomatic alternative way of doing this.

logrotate is available, but I'd rather avoid doing a rotation every because it seems "simpler" to have it all done in step.

multilog seems to tackle a similar problem of switching target log file, but when the log file is full rather than when the date changes. I don't have root access and can't run a package manager to get hold of it; I haven't investigated whether I could build it from source successfully using the permissions that I have. If it's clearly the tool for the job then I'll pursue this option.

  • 1
    What about -f $(date +"%Y-%m-%d") – Lambert Feb 17 '17 at 11:11
  • I assume that's a comment about how I'd make the "pushtolog"'s -f argument contain the current date? Unfortunately the "pushtolog" command doesn't exist! I'm looking for a suggestion of how to implement the whole last piece of the pipeline; to replace this with something real: "pushtolog -d /tmp/myerrors/ -f yyyy-MM-dd -r errors -e log" In my imaginary example, I set the date format to year-month-day using the -f switch. I don't want that to be evaluated once today, because it needs to take a different value tomorrow. The utility needs the date format, not the current date. – David B Feb 17 '17 at 11:17
  • And what is the meaning for -r and -e? – Lambert Feb 17 '17 at 11:28
  • Sorry that it isn't clear. -r is "root of the filename", which in my example is "errors" and -e is "extension", which in my example is "log". Putting together the root, date and extension, you get "errors.2017-02-17.log" as the output filename. – David B Feb 17 '17 at 11:30
  • I think I'll end up doing something with AWK, like in this example: stackoverflow.com/questions/13077631/… – David B Feb 17 '17 at 11:33
0

You can try the following code, as a bash script:

#!/bin/bash

while getopts "d:f:e:r:a" opt; do
    case $opt in
        f) FDATE=$(awk -v fmt="$OPTARG" '{print strftime(fmt)}'<<<DUMMY) ;;
        r) PREFIX="$OPTARG" ;;
        e) SUFFIX="$OPTARG" ;;
        d) DEST="$OPTARG" ;;
        a) APPEND=true ;;
    esac
done
if [ ! $APPEND ]
then
    > ${DEST}/${PREFIX}.${FDATE}.${SUFFIX}
fi
while read line; do echo ${line} >> ${DEST}/${PREFIX}.${FDATE}.${SUFFIX}; done

You can also surround the body of the script with:

pushtolog()
{
...
}

and source it.

I also included a flag to be able to append to existing data. It should be easy to change the script to default to append and overwrite when a specific option is set.

Note 1: The date format should conform to strftime (see man strftime) so use "%Y-%m-%d"

Note 2: that variable APPEND is not actually a boolean, I just imagine that it is. Perhaps it is more nice to use APPEND="true" and if [ -z "$APPEND" ]

0

I used Awk's ability to evaluate construct a filename including today's date on each evaluation (for each line that orignates from the log) and ended up with:

tail -F mylatestlogfile.log | grep "ERROR" | awk '{print >> "/tmp/myerrors/errors." strftime("%Y-%m-%d") ".log"}'

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.