Can I migrate ubuntu journald logs?

Question

I have an app registered as a service in Ubuntu 16.04

If I type:

journalctl -u myapp.service

I can see the logs for my app.

I am moving my app to a new VM where the same service will be in place. Is it possible to migrate the log files to the new VM so that if I type journalctl -u myapp.service it will show all my old logs and any new logs seamlessly?

I have tried to swap in the contents of the old /var/log/journal directory into the new VM, and restarting the systemd-journald service, but it doesn't seem to work.

More Details:

• The logs are stored in /var/log/journal/< machine-id >/
• the directory contents look like this:

$ ls /var/log/journal/05b6b1e76c6040cc99b4d34977a98eca/ [email protected]~ [email protected]~ [email protected]~ [email protected]~ system@9b08b416ae4c47a78c24b4ed77c39ea2-0000000000000001-0005b3c2d1380bae.journal system@9b08b416ae4c47a78c24b4ed77c39ea2-0000000000000248-0005bccbf3f7d7c1.journal system@e5c655526bb54aa886764039cd37f897-0000000000000001-0005c4ce02f66caf.journal system.journal [email protected]~ [email protected]~ [email protected]~ [email protected]~ user-1000@7b4df282ccfe4816a30db088f2621493-00000000000000ab-0005b3c2d1be9c3b.journal user-1000.journal

• There are no files in /run/log/
• both old and new VMs share the same service, service-running user, and machine-id (this is because ultimately both VMs stem from (were copied from) the same VM just at different times and with minor software/configuration settings)
• the user I access the logs with 'ubuntu' has the same groups in both VMs:

$ groups

ubuntu adm dialout cdrom floppy sudo audio dip video plugdev netdev lxd

• journal version on both VMs is:

$ journalctl --version

systemd 229

+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN

• I am not looking to hold the logs longer than what is configured in the journal settings

Whats Working

• replacing the contents of the journal folder entirely
• adjusting file ownership to root:systemd-journal as they were originally
• 'root' can now see the full log
• 'ubuntu' can only see some 'current' logs, which are confusing, as they are not what was copied over

Answer 1

All entries for which you will find a relation using journalctl -u <myapp.service> correspond to the contents of the systemd journal for a given host. HOWEVER (from man journalctl):

Output is interleaved from all accessible journal files, whether they are rotated or currently being written, and regardless of whether they belong to the system itself or are accessible user journals.

The issue may boil down to the fact that on the "new VM" (a remote PaaS for instance, or even a VM on a local host) your $USER on that VM may not be granted access to anything but his/her own, private journals. For any other journals (provided your migrated log files are located in the right place) your $USER will need to be part of a few special authorized groups or to be root.

That would apply to any imported journal files since those would not be recognized as your $USER's private journals on the new host. Special groups include systemd-journal, adm, and wheel. Ubuntu may include others.

To access your migrated files you should start making sure that:

• you migrated your journals to the single central journal in /{run,var}/log/journal/. I'm not certain which of /run or /var begins the path on your Ubuntu VM; you will have to see that by yourself. In any case see journald.conf(5), by issuing $ man 5 journald.conf in a terminal. You will find that your files may be at:

  /run/log/journal/machine-id/*.journal,
  /run/log/journal/machine-id/*.journal~,
  /var/log/journal/machine-id/*.journal,
  /var/log/journal/machine-id/*.journal~
  

From the Ubuntu man page:

systemd-journald writes entries to files in either /run/log/journal/machine-id/ or /var/log/journal/machine-id/ with the ".journal" suffix. If the daemon is stopped uncleanly, or if the files are found to be corrupted, they are renamed using the ".journal~" suffix, and systemd-journald starts writing to a new file.
[...]
/run/... is used when /var/log/journal is not available, or when Storage=volatile is set in the VM's /etc/systemd/journald.conf configuration file.

• your VM's $USER is member of the wheel group (for instance).

Other recommendations and steps may apply but I do not have enough info to run on to proceed. Please report in a comment, so I may complete this answer if need be.