24

I spent a few days writing a python script, and creating a systemd unit file for it. During testing, the script logged a lot of errors to journald. I would like to clear those errors from journald now that I'm done.

There are several ways to clear the entire journal, as described here: How to clear journalctl including using journalctl --vacuum-time=2d, using journalctl --vacuum-size=500M, and temporarily setting SystemMaxUse= in /etc/systemd/journald.conf to a very low value.

All of these appear to clear the entire journal, effecting all units. I just need to clear the entries for a single unit. Is this possible?

1
10

Use my Python 3 program copy_journal.py on the journal files in /var/log/journal from which you want to remove entries.

For instance, to make a copy of system.journal without log entries for NetworkManager.service:

$ journalctl --file=system.journal | wc
    167    1934   18825
$ journalctl --file=system.journal | grep -v NetworkManager | wc
     77     881    8421
$ python3 copy_journal.py --remove-unit=NetworkManager.service system.journal system-without-nm.journal
$ journalctl --file=system-without-nm.journal | wc
     77     881    8421
1
  • Wow, huge work on this package! – Joël May 31 '20 at 8:45
1

I was looking for a way to remove a specific message from the systemd journal. This didn't seem like an uncommon scenario (e.g. accidentally running a wrong command or having lots of spam).

It appears to be infeasible to redact specific entries by modifying an existing file, because the systemd Journal format is primarily designed to support appending, which involves optional compression, hashing, updating counters, indexing references to offsets and hashes. I tried this at first, but would not recommend anyone to do this.

To remove specific entries, I found that the next best thing is to generate a new log file based on the existing log file (e.g. the Python script from the other answer does this). Systemd actually ships the necessary tools to easily modify log entries using this principle.

Short version

In essence, export the journal to a text format, and then convert it back to a journal:

  • journalctl --file=system.journal -o export | /lib/systemd/systemd-journal-remote --output=result.journal

This mechanism can easily be extended by piping the initial output to a program/script (such as awk, sed, etc.) to do automatic filtering before piping it to systemd-journal-remote.

A more manual approach is to write the text to a file, modify the file by hand with a text editor and then converting it back to a .journal file. This is explained in more detail below.

Long version

Here is the longer version, a step-by-step guide to modify an existing journal (tested on ArchLinux, using an Ubuntu live USB, systemd 246).

  1. First, ensure that the journal has been closed. For example by shutting down your computer.

  2. Boot a different device (via a bootable live USB for example). This is to be able to make changes without inadvertently impacting the journal.

  3. To minimize accidental side effects, mount the original disk as read-only. For example:

    $ sudo mount -o ro /dev/sda1 /mnt
    

    another example, if you're using LUKS to encrypt the device:

    $ sudo cryptsetup luksOpen /dev/sda1 whateverName
    $ sudo mount -o ro /dev/mapper/whateverName /mnt
    
  4. If your current device doesn't have systemd / journalctl, use a chroot to be able to use the binaries from your read-only disk. The minimum to get this to work is as follows:

    $ sudo mount --bind /tmp /mnt/tmp
    $ sudo mount --bind /tmp /mnt/var/tmp
    $ sudo mount -t proc /proc /mnt/proc
    $ sudo chroot /mnt
    


  1. When you are ready, export the journal.

    $ journalctl --file=system.journal -o export > /tmp/export.txt
    
  2. Edit the export to redact, add, modify or remove entries. You can do this outside the chroot if you want to, e.g. with vi -b /tmp/export.txt.
    Example of the output: https://www.freedesktop.org/wiki/Software/systemd/export/

  3. When you have made the desired modifications, convert it back to a journal using systemd-journal-remote. This binary is not in bin / $PATH, so you need to use the absolute path to the program's location:

    $ /lib/systemd/systemd-journal-remote --output=/tmp/result.journal /tmp/export.txt
    
  4. Verify that the journal has the desired output, e.g. by paging through the journal with:

    $ journalctl --file=/tmp/result.journal
    

    If you are satisfied, proceed with the next step. Otherwise repeat from step 6.



  1. Exit the chroot (CtrlD or simply exit) and undo step 4, if needed:

    $ sudo umount /mnt/proc
    $ sudo umount /mnt/tmp
    $ sudo umount /mnt/var/tmp
    
  2. Now we are about to overwrite the original journal.
    If you have mounted the volume as read-only (step 3), remount as writable.

    $ sudo mount -o remount,rw /mnt
    

    To avoid unwanted changes in file attributes or ownership, just replace the content instead of copying the file.

    # cat /tmp/result.journal > /mnt/var/log/journal/MACHINE_ID_HERE/system.journal
    

    or if you aren't running in a root shell, use sudo tee outfile >/dev/null, like this:

    $ </tmp/result.journal sudo tee /mnt/var/log/journal/MACHINE_ID_HERE/system.journal >/dev/null
    
  3. Unmount the disk (from step 3 / step 10) if needed.

    $ sudo umount /mnt
    

    If you had opened an encrypted volume (at step 3), lock it again:

    $ sudo cryptsetup luksClose whateverName
    
0

I do not think think its possibble, because according to journalctl man page "Output is interleaved from all accessible journal files, whether they are rotated or currently being written".

You could however filter it by a unit - 'journalctl -u some.service' and then try to remove the log entries from the journal with your own script using cron.

1
  • 8
    It sounds like you are implying that there is a way to delete individual log entries from journald. Is there one? That would be very helpful. – Tal Mar 29 '16 at 16:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.