4

By default my system had about ~500MB max size which had about 1 month of logs.

System Journal (/var/log/journal/dd35c7606a5645c5acc9908470c45159) is 483.2M, max 491.5M, 8.2M free.

No idea how the 491.5M limit was decided. I changed the below setting and restarted the system

└──> grep SystemMaxUse /etc/systemd/journald.conf
SystemMaxUse=25G

Now the max journal size is 1.0G. Why is it not 25G?

System Journal (/var/log/journal/dd35c7606a5645c5acc9908470c45159) is 568.6M, max 1.0G, 455.3M free.

I do have enough free space (94%) in the disk

└──> df -h /var/log/journal
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       885G   50G  791G   6% /var

How to increase max journal size even further?

If possible, I would like to keep the default limit for 1 month but instead of deleting old journal files, save it some where where in can be queried on demand.

Update: Just to clarify, my journal files are now deleted after 500MB cap, I want to increase the limit to 25G. Even after setting SystemMaxUse=25G, the limit does not increase past 1GB.

6
  • Editing the config file worked for me. Did you restart systemd-journald.service? Maybe post the entire journald.conf file? Commented Jul 12, 2022 at 17:19
  • yes. After restarting, the max changed from 429.5M to 1.0G. That is the entire journald.conf, rest of them are commented out by default
    – balki
    Commented Jul 12, 2022 at 18:00
  • 1
    In addition to SystemMaxUse the documentation also lists SystemKeepFree (which you may want to minimize) and SystemMaxFileSize for individual file limits (which you may need to increase?). The documentation also indicates that the values may be capped, somehow, so studying the source code might be necessary.
    – thrig
    Commented Jul 12, 2022 at 18:16
  • SystemKeepFree, is supposed to default to 15%, but I have more than 90% free in the filesystem. Not sure why the size is capped to 1.0G.
    – balki
    Commented Jul 12, 2022 at 19:32
  • What distro/version is this on? It might help track down the issue since it could be a default security feature limiting your journal file size.
    – ReedGhost
    Commented Jul 29, 2022 at 20:56

4 Answers 4

4

The systemd-journald options are fully documented at Freedesktop.org

In this case the options you are looking for are: SystemMaxUse, SystemKeepFree, SystemMaxFileSize, and SystemMaxFiles. You've set SystemMaxUse which is the total limit for persistent journals including rotation across boots. Your problem is that your uptime is so long you are exceeding the limit for a single journal. So you should set SystemMaxFileSize to 25GB too so that very large single journals are allowed.

In addition you should set SystemMaxFiles to something large like 1000 so that you can reset almost 3 times a day and still not have the first day of logs in a year deleted unless you run out of allocated space.

Additionally you should set MaxRetentionSec to 31536000 to get 365 days of retention as mentioned above.

Essentially the conditions for log retention have to all be and'ed together for a log to be kept so any applicable rule applies and needs to be set. As a result even this answer may be incomplete.

1
  • 1
    Basically the most limiting setting (time limit, disk space, file count, file size) limits the amount of log you'll have. You have to increase every setting to match your needs because default limits are pretty small (except for file size which can grow up to 4 GB by default). Commented Dec 5, 2023 at 11:44
0

There is the exactly desired option in the journald.conf file.

From the man page [1]:

MaxFileSec=

    The maximum time to store entries in a single
    journal file before rotating to the next one.
    [...]
    This setting takes time values which may be
    suffixed with the units "year", [...]

[1] https://www.freedesktop.org/software/systemd/man/journald.conf.html#MaxFileSec=

1
  • 2
    Yes. But if size is anyway capped at 1.0G, it is not going to save for 1 year System Journal (/var/log/journal/dd35c7606a5645c5acc9908470c45159) is 568.6M, max 1.0G, 455.3M free.
    – balki
    Commented Jul 28, 2022 at 13:19
-1

set up a cronjob to run on a periodic basis, e.g for your use case, run this daily:

$ journalctl --vacuum-time=1y

refs:

-1

No idea how the 491.5M limit was decided. I changed the below setting and restarted the system

AND:

By default my system had about ~500MB max size which had about 1 month of logs.

AND:

Now the max journal size is 1.0G. Why is it not 25G?

ANSWER TOO THOSE QUESTIONS:

Reinout van Schouwen 2015-11-05 20:42:54 CET

Description of problem: I got warnings about disk space on my root partition. The cause was /var/log/journal with gigabytes of logs.
All options in /etc/systemd/journald.conf are commented out so journalctl doesn't keep a certain maximum size. We should be smarter about this so that logs don't fill up the root partition.

AND:

David Walser 2015-11-06 15:18:21 CET

I thought systemd would only allow it to use a certain % of disk space on its partition. If it's really unbounded, that's a problem.

AND:

Colin Guthrie 2015-11-09 10:44:45 CET

So in order to investigate this any further, I'll need information about your system. What is your partition size, what is the free space etc. A listing of the files and folders inside /var/log/journal would also be useful (including their full path, dates and sizes). Also the output from "journalctl --disk-usage" (as root) and the output from "journalctl -b MESSAGE_ID=ec387f577b844b8fa948f33cad9a75e6" (also as root).

SOURCE:

https://bugs.mageia.org/show_bug.

ANSWER TOO:

How to increase max journal size even further?

Systemd Journals Logging with Persistent Storage

Configuring the systemd journal for persistence in RHEL7

Configuring the systemd journal for persistence in RHEL7 By default, the journal doesn't store log files on disk, only in memory or the /run/log/journal directory. This is sufficient for the recent log history (with the journal) but not for long-term log retention should you decide to go with journal only and not with any other syslog solution.

SOURCE3:

https://www.linvirtshell.com

or

Managing Journal Size

By default, systemd-journald ensures older journal records or journal files are deleted in order to keep a certain amount of disk space free. In the Linux Logging with Systemd section, we explain how to control the thresholds for deleting old log data with configuration parameters. We can also use journalctl to manage the size of the journal, which we’ll explain in more detail in this section.

To check how much disk space is currently taken up by the journal, use the –disk-usage parameter:

$ journalctl --disk-usage

Depending on the version of journalctl, the output can be similar to this:

Archived and active journals take up 96.0M on disk.

or:

Journals take up 248.0M on disk.

You can also manage disk space taken up by systemd journal by fine-tuning these configuration parameters:

COMPRESS
SYSTEMKEEPFREE
RUNTIMEKEEPFREE
SYSTEMMAXFILESIZE
RUNTIMEMAXFILESIZE
MAXRETENTIONSEC
MAXFILESEC

SOURCE:

https://www.loggly

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .