I have a systemd service that I install via debian package using the
apt command on a raspberry pi 4. I've been using
journalctl -xef to keep track of all log output for debugging purposes while installing the package and running the service.
I'm not sure how I managed this, but at some point today,
apt's log output as well as the output of my package's maintainer scripts stopped showing up in the journal. I did have several typos in one of my package's maintainer scripts and I have a tendency to fat finger the keyboard or press the hotkeys sometimes, but I can't find anything in particular that might've caused the issue. It seems that the journal defaults to in-memory storage as well, which is a bummer, because the first thing I tried to fix the issue was a reboot. So now I guess I can't even look back at old journal entries to see if there was something in there that might indicate what had happened.
I also thought perhaps some of the changes I'd made today in the service or package were causing the problem, so I tried installing an older proof-of-concept service/package and that too does not write correctly to the journal.
I've tried to find a way to just reinstall or repair the
apt/apt-get/dpkg installation, but haven't been very successful. Most questions are related to repairing a package installation using
apt, not repairing the installation of
apt itself. I also tried to see if there was a way to configure where
apt writes its logs, but most questions along those lines are related to redirecting
stdout/stderr when running an
I'm very new to linux and all of the technologies listed above, so I'm not sure how to debug/diagnose this further.
Is there a way to repair/restore the
apt/apt-get/dpkg installations or configurations on a raspberry pi or is my only option to try to restore to factory defaults or re-install the entire OS?
Today I learned of the
/var/log directory. I haven't found any indication within those logs of what caused this issue, but it does appear that the the problem also extends to
daemon.log (i.e. output from
apt/apt-get and from package maintainer scripts is missing from these as well). I can see several
apt install/upgrade/purge operations successfully logged in those files and then no more. I can tell there should be more entries logged, however, because my package calls
systemctl daemon-reload and I can see
systemd: Reloading in the logs. From browsing
/var/log/dpkg.log, it seems that none of those entries were every shown in
journald, so that seems to still be fine. Logs from
/var/log/apt/term.log are definitely missing from
journald, but logs from
/var/log/auth.log are still present.