1

I have an array of load balanced app servers (app1-app4) for multiple websites logging to a single log server (log1). I would like the logs on the log server to be grouped by virtualhost so I can see can get a single logs for each VirtualHost.

Currently I have my apache sending logs to my rsyslog server with the VirutalHost as the leading field, like:

LogFormat "%V %h %l %{USERID}e %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" \"%{Cookie}n\""  combinedcookie
CustomLog "|/usr/bin/logger --size 8192 -t httpd -p local6.info" combinedcookie

Creates combined local logs on each appserver like:

www.example.com 190.120.94.32 - - [19/Jun/2020:14:47:05 -0400] "GET /help/ HTTP/1.1" 200 6621 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" "*cookie*"
fubar.pro 177.27.44.23 - guest [19/Jun/2020:14:52:06 -0400] "GET /login HTTP/1.1" 200 6746 "https://www.google.com/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" "*cookie*"
www.yadayada.ca 176.82.169.235 - user [19/Jun/2020:14:55:04 -0400] "GET / HTTP/1.1" 200 5716 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" "*cookie*"

Log server rsyslog.conf looks like:

$template HttpdTmp,"/backup1/logs/www/%HOSTNAME%/%$YEAR%/live_access_log-%$YEAR%%$MONTH%%$DAY%"
$template RawMsgFormat,"%msg:2:4096%\n"
local6.*                -?HttpdTmp;RawMsgFormat

Which creates logs like:

/backup1/logs/www/app1/2020/live_access_log-20200619
/backup1/logs/www/app2/2020/live_access_log-20200619
/backup1/logs/www/app3/2020/live_access_log-20200619
/backup1/logs/www/app4/2020/live_access_log-20200619

What I would like on the log server is:

/backup1/logs/www/www.example.com/2020/live_access_log-20200619
/backup1/logs/www/fubar.pro/2020/live_access_log-20200619
/backup1/logs/www/yadayada.ca/2020/live_access_log-20200619

Is there some magical config that can be done with rsyslog tags or similar? Or can I split based on the first word in log (the VirtualHost name)?

(*IP's have been randomized to protect the anonymous)

1
  • 1
    FYI, logger needs --size parameter > 1024 (default) because it is not uncommon for referring URL's to push the log string over 1K. make sure you have a recent logger as support for that parameter is recent in the RedHat ecosystem.
    – Stickley
    Jun 29, 2020 at 14:10

1 Answer 1

1

Assuming the string www.example.com is at the start of the msg property, you want to replace the %HOSTNAME% in the filename template by that value. Probably the simplest property replacer operation is to extract a field delimited by a space. This has the form

%key:F,delimeter:numfield:options%

where the delimiter is the character code in decimal, i.e. 32 for space, and fields begin from 1 for the first. As it seems you have a space at the beginning of msg you might need to select the 2nd field and so use:

%msg:F,32:2%

So for your example, the log server rsyslog.conf will look like:

$template SiteTmp,"/backup1/logs/www/%msg:F,32:2%/%$YEAR%/live_access_log-%$YEAR%%$MONTH%%$DAY%"
$template RawMsgFormat,"%msg:2:4096%\n"
local6.*                -?SiteTmp;RawMsgFormat
1
  • This worked right of the box. Thank you!
    – Stickley
    Jun 23, 2020 at 19:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .