0

One of our Ubuntu 18.04 hosts was caught with 12 GB of *.journal files, far more than intended. Attempting to find out if they were worth keeping, I ran

journalctl --file $f

on each file older than today; which always resulted in either Failed to open files or --- No entries ---.

Am I correct to conclude that such files are junk and can be discarded?

If they are, why do they exist? What is a supported way to clean them up? Is it worthwhile to regularly check systems for their existence?

1

First of all Journal is a logging system and is part of systemd. Their existence is crucial when you need to know what happened.

As mentioned here, journalctl --file isn't that usable.

As the journal files are rotated periodically, this form is not really usable for viewing complete journals.

Now, whether you consider the files useless, that's for you to decide. Normally, too old logs are not worth keeping and you could delete them.

To do that, is best to use journalctl itself and its utility vacuum. For instance you can use

sudo journalctl --vacuum-time=3weeks

to delete all journal files that are more than 3 weeks old.

For more info check the man page with man journalctl.

--vacuum-size=, --vacuum-time=, --vacuum-files=

Removes the oldest archived journal files until the disk space they use falls below the specified size (specified with the usual "K", "M", "G" and "T" suffixes), or all archived journal files contain no data older than the specified timespan (specified with the usual "s", "m", "h", "days", "months", "weeks" and "years" suffixes), or no more than the specified number of separate journal files remain. Note that running --vacuum-size= has only an indirect effect on the output shown by --disk-usage, as the latter includes active journal files, while the vacuuming operation only operates on archived journal files. Similarly, --vacuum-files= might not actually reduce the number of journal files to below the specified number, as it will not remove active journal files.

Also, I don't believe its worthwhile to periodically check this. Best thing you can do is set an upper limit by uncommenting and changing the following in /etc/systemd/journald.conf.

For example:

SystemMaxUse=4G

Then restart the service. sudo systemctl restart systemd-journald.

Use man journald.conf for more information.


Edit:

As explained by @reinierpost

This question is not about regular old logs, it is about old logfiles that do not appear to contain any logs at all (but they still occupy 8 MB each).

Try running journalctl --verify. If files don't pass then the journal is corrupted and you should restart the service.

sudo systemctl restart systemd-journald

That should fix the problem for logs going forward.

As for why this happened in the first place, I don't know and its not easy to figure out. And yes, corrupted files are probably junk. You could try this for a clean slate.

  • SystemMaxUse is and was set to 250M; clearly, that doesn't always work. Maybe systemd-journald hadn't been restarted since it was set, I didn't check that. I didn't try the vacuuming utility. This question is not about regular old logs, it is about old logfiles that do not appear to contain any logs at all (but they still occupy 8 MB each). If I understand correctly, journalctl --file should show log entries in old rotated logs if any entries are in there. – reinierpost Nov 19 '20 at 15:49
  • Yes, I forgot to mention it, you should restart the service for the changes to take effect. As for the logs try running journalctl --verify. If files don't pass then the journal is corrupted and you should restart the service. That should fix the problem for logs going forward. As for why this happened in the first place, I don't know and its not easy to figure out. And yes, corrupted files are probably junk. You could try this for a clean slate. If that answers your question please let me know, so I can update the answer. – Rayleigh Nov 19 '20 at 19:05
  • I've removed the files, so I cannot try it anymore, but I will if it reoccurs, thanks. – reinierpost Nov 19 '20 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.