2

TL;DR how comes that journald stores about 770 MiB while displaying the logs only yields 90MiB of data?

#UPDATE# I did a journalctl -a -o verbose which at least yielded some 530MiB, yet less than the 770MiB the journal uses on disk. Anyway I would have expected that the journal on disk is compressed in some sort, and should be less than the 530MiB and not bigger instead. what is going on?

#UPDATE2#

It seems that journald has an tendency to create mostly empty journal files.

$> cat .system@0000.journal~ | wc -c
8388608
#> cat .system@0000.journal~ | sed 's/\x00//g' | wc -c
47181

which might be what the comment by @Mioriin hints at. Is there any reason why there are 8MiB , but almost empy journal files kept by journald?

long version with background

My system is quite chatty, with linux kernel dmesg output, gnome DE, etc. - filling up my /var/log directory via the system logger journald that I sadly use (being still on a standard systemd arch linux).

I looked via du -hs /var/log and found roughly 2GiB with most being of journald journals. I removed the data being older than 90 days via journalctl --vacuum-time=90d and alas with sacrfificing logs older than ~3 month reduced the disk space used for the journald logs to only about 770MiB as checked via

$> journalctl --disk-usage 
Archived and active journals take up 768.0M in the file system.

which is great. However 770MiB of data for 90 days on my arch linux box seems quite a bit so I tried investigating to see what for the 770MiB are used for. I did the imho first logical step to measure initial size of the log output via

$> journalctl | wc
584467 7662317 90227741

and to double check did

$ journalctl | dd of=/dev/null iflag=fullblock bs=1M
86+1 records in
86+1 records out
90244229 bytes (90 MB, 86 MiB) copied, 9.21893 s, 9.8 MB/s

showing that for some reason, reading the "binary-because-saves-space" journald in plain text is not yielding some < 770MiB of data but instead is only 90MiB.

Now my question is about why those two figures do not match? How can my beloved journald, end up using 770MiB for essentially storing 90MiB of data?

What, next to "the meauring and being confused about the result", have I done to help myself?

I have skimmed the lengthy man journactl but am clearly at a loss with the considerable intricacy and detail it offers, yet could not find an hint as to what might cause the counter-intuitive thing that the binary-compressed-journal-data uses more space than the actuall plain text reproduced log.

I guess there should be an "easy", explanation.

  • @don_crissti thanks for pointing to a similar question on serverfault. I checked it and the provided answer, while useful in the investigation, do not provide an answer. It seems that journald when allocating disk spaces uses multiple of 8MiB sized files, even when only (as in one case of mine) 56kiB of journal -o verbose is stored stored in this file. – humanityANDpeace Aug 13 '18 at 11:23
  • Have a look inside /var/log/journal. That should clarify things a bit for you. ;) – Mioriin Aug 13 '18 at 11:57
  • @Mioriin do you imply this github.com/systemd/systemd/issues/5285 issue? what do you mean. – humanityANDpeace Aug 13 '18 at 11:58
  • journalctl by default only loads the current journal file and not all the archived journals to save on resources. If it were to load all 90 days' worth of logs, you'd be waiting quite a while for anything to show up. To verify that it's actually using ~770MiB, try du -h /var/log/journal. – Mioriin Aug 13 '18 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.