Assume, that beside the Apache web server logs I never had any contact with any kind of (professional) logs on any operation system. So Logging, although I understand some basics, is all together a pretty new topic. At the moment the investment to fully learn about this topic seems to be quite huge, yet I don't even know yet, if it is even worth knowing more then the most abstract concepts.

Which resources would you suggest should someone in that situation consume (tutorials, man pages, books) to learn about Logging?

Which logs should a normal Linux user read on a daily/monthly basis? Is the assumption even correct that they are written for human readability or are they generally evaluated and used by other tools?

What should the normal *nix user and software developer know about these logs?

What do you need to know about log rotation, if you are not expected to manage professional web servers with huge loads of events?

  • Do you mind if I (or you) edit the title to read "on linux" instead of "on ubuntu" since these are essentially the same question (see my answer)? That way we are less likely to end up with closed duplicate questions on "learning logging on arch", "learning logging on centOS", etc.
    – goldilocks
    Feb 27, 2013 at 15:02
  • I don't mind such kind of changes, as long as the answers are good :)
    – erikbstack
    Mar 1, 2013 at 17:09

1 Answer 1


[This was written a few years before the widespread adoption of journald on systemd systems and does not touch on it. Currently (late 2018) both journald and (r)syslog, described below, are used on distros such as Debian. On others, you may have to install rsyslog if you want to use it alongside, but the integration with journald is straightforward.]

I won't discuss logging with regard to ubuntu specifically much, since the topic is standardized for linux in general (and I believe most or all of what I have to say is also true in general for any flavor *nix, but don't take my word for that). I also won't say much about "how to read logs" beyond answering this question:

Is the assumption even correct that they are written for human readability or are they generally evaluated and used by other tools?

I guess that depends on the application, but in general, at least with regard to what goes into syslog (see below), they should be human readable. "Meaningful to me" is another issue, lol. However, they may be also be structured in a way that makes parsing them with standard tools (grep, awk, etc) for specific purposes easier.

Anywho, first, there is a distinction between applications which do their own logging and applications which use the system logger. Apache by default is the former, although it can be configured to do the later (which I think most people would consider undesirable). Applications which do their own logging could do so in any manner using any location for the file(s), so there is not much to say about that. The system logger is generally referred to as syslog.


"Syslog" is really a standard that is implemented with a daemon process generically called syslogd (d is for daemon!). The predominant syslog daemon currently in use on linux, including ubuntu, is rsyslogd. Rsyslogd can do a lot, but as configured out of the box on most distros it emulates a traditional syslog, which sorts stuff into plain text files in /var/log. You might find documentation for it in /usr/share/doc/rsyslog-doc-[version] (beware, there is also a /usr/share/doc/rsyslog-[version], but that's just notices from the source package such as NEWS and ChangeLog). If it's there, it's html, but Stack Exchange doesn't permit embedding local file links:


So you could try copy pasting that. If it's not there, it may be part of a separate package that is not installed. Query your packaging system (eg, apt-cache search rsyslog | grep doc).

The configuration is in /etc/rsyslog.conf, which has a manual page, man rsyslog.conf, although while the manual page makes a fine reference, it may be less penetrable as an introduction. Fortunately, the fundamentals of the stock rsyslog.conf conform to those of the traditional syslog.conf, for which there are many introductions and tutorials around. This one, for example; what you want to take away from that, while peering at your local rsyslog.conf, is an understanding of facilities and priorities ("priority" is sometimes referred to as loglevel), since these are part of the aforementioned syslog standard. The reason this standard is important is because rsyslog actually gets its stuff via the kernel, and what the kernel implements is the standard.

With regard to the $ directives in rsyslog.conf, these are rsyslog specific and if you install that optional doc package you'll find a guide to them in rsyslog_conf_global.html.

Have fun...if you are curious about how applications use the system logger, look at man logger and man 3 syslog.

Log Rotation

The normative means of rotating logs is via a tool called logrotate (and there is a man logrotate). The normative method of using logrotate is via the cron daemon, although it does not have to be done that way (e.g., if you tend to turn your desktop off everyday, you might as well just do it once at boot before syslog starts but, obviously, after the filesystem is mounted rw).

There's a good introduction to logrotate here. Note that logrotate is not just for syslog stuff, it can be used with any file at all. The base configuration file is /etc/logrotate.conf, but since the configuration has an "include" directive, commonly most stuff goes into individual files in the /etc/logrotate.d directory (here d is for directory, not daemon; logrotate is not a daemon).

An important thing to consider when using logrotate is how an application will re-act when its log file gets "rotated" -- in other words, moved -- while the application is running. WRT (r)syslogd, it will just stop writing to that log (I think there is a security justification for this). The usual way to deal with that is to tell syslog to restart (and re-open all its files), which is why you will see a postrotate directive in logrotate conf files sending SIGHUP to the syslog daemon.

  • 2
    If you mention syslog-ng, too, you will have written all there is to say on linux-logging. Excellent answer.
    – Nils
    Feb 27, 2013 at 20:28
  • I learned a lot and have more sources to read. Thanks! Btw. Why would it be bad to use system logging for apache? And which logs should I want to read?
    – erikbstack
    Mar 1, 2013 at 18:16
  • @erikb85 1) Mostly because apache generates a lot of output. I have not had to configure apache in a long time, so I can't remember how flexible it is WRT to sending specific things to syslog (but I'd guess very flexible) 2) WRT logs you want to read, have a look at the logic in your /etc/rsyslog.conf. For example: often stuff above a certain priority will go into /var/log/messages, and stuff below that will go into /var/log/notice. I like to also have a log that contains everything, which makes for double and triple overlap, but if you keep them rotated it is no big deal.
    – goldilocks
    Mar 2, 2013 at 16:22
  • Just to confirm, rsyslog is not a daemon wrapper around logrotate, right ? Why so ?
    – Thomas
    Nov 20, 2014 at 10:54
  • @Thomas They perform different purposes, although (r)syslog could implement rotation to spare the need for logrotate. However, having them separate means you can use logrotate on things other than syslog files.
    – goldilocks
    Nov 20, 2014 at 13:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .