317

I have created a simple systemd service file for a custom application. The application works well when I run it manually, but my CPU gets maxed out when I run it with systemd.

I'm trying do track down where my problem is, but I don't know where to find the output (or how to configure systemd to put the output somewhere).

Here is my service file:

[Unit]
Description=Syncs files with a server when they change
Wants=network.target
After=network.target

[Service]
ExecStart=/usr/local/bin/filesync-client --port 2500
WorkingDirectory=/usr/local/lib/node_modules/filesync-client
Restart=always

[Install]
WantedBy=multi-user.target

Throughout the application, I output to stdout and stderr.

How can I read the output of my daemon?

Edit:

I found man systemd.exec, which mentioned the StandardOutput= option, but I'm not sure how to use it. From the man page:

StandardOutput=

Controls where file descriptor 1 (STDOUT) of the executed processes is connected to. Takes one of inherit, null, tty, syslog, kmsg, kmsg+console, syslog+console or socket.

If set to inherit the file descriptor of standard input is duplicated for standard output. If set to null standard output will be connected to /dev/null, i.e. everything written to it will be lost. If set to tty standard output will be connected to a tty (as configured via TTYPath=, see below). If the TTY is used for output only the executed process will not become the controlling process of the terminal, and will not fail or wait for other processes to release the terminal. syslog connects standard output to the syslog(3) system logger. kmsg connects it with the kernel log buffer which is accessible via dmesg(1). syslog+console and kmsg+console work similarly but copy the output to the system console as well. socket connects standard output to a socket from socket activation, semantics are similar to the respective option of StandardInput=. This setting defaults to inherit.

Does this mean that these are my only options? I would like, for example, to put output in /dev/shm or something. I suppose I could use a Unix domain socket and write a simple listener, but this seems a little unnecessary.

I just need this for debugging, and I'll probably end up removing most of the logs and change the output to syslog.

Improve this question
7
  • 1
    Have you tried checking /var/log/syslog for output? Most systems will log stuff into /var/log/ so I'd start by checking there. You can use grep to search for text if you know the output: grep "my output" /var/log should do the trick.
    – sbtkd85
    Commented Sep 9, 2011 at 19:10
  • @sbtkd85 - Well, I don't have /var/log/syslog, but /var/log/messages does the trick. The problem is, according to the logs, my daemon crashes on start, yet I can tell that it is still running because it has an HTTP server, and I can query it. It seems the rest of the logs are getting lost...
    – beatgammit
    Commented Sep 10, 2011 at 3:50
  • Why not try setting StandardOutput=tty so you can see what is happening when you launch your daemon. It should output the terminal (you may have to use ttyS0 or similar to get the output on your screen).
    – sbtkd85
    Commented Sep 12, 2011 at 13:46
  • 5
    Shouldn't standard IO redirection operators work in this context. Something like ExecStart=/usr/local/bin/filesync-client --port 2500 2>/tmp/filesync.log
    – Deepak Mittal
    Commented Apr 10, 2012 at 18:30
  • What is actually abusing your CPU? Is it systemd, your service or system (e.g. by spawning new copies of the service because systemd went crazy)?
    – peterph
    Commented Nov 13, 2012 at 16:03

2 Answers 2

Reset to default
320

Update

As mikemaccana notes, the systemd journal is now the standard logging device for most distros. To view the stdout and stderr of a systemd unit use the journalctl command.

sudo journalctl -u [unit]

Original Answer

By default stdout and stderr of a systemd unit are sent to syslog.

If you're using the full systemd, this will be accesible via journalctl. On Fedora, it should be /var/log/messages but syslog will put it where your rules say.

Due to the date of the post, and assuming most people that are exposed to systemd are via fedora, you were probably hit by the bug described here: https://bugzilla.redhat.com/show_bug.cgi?id=754938 It has a good explanation of how it all works too =) (This was a bug in selinux-policy that caused error messages to not be logged, and was fixed in selinux-policy-3.10.0-58.fc16)

Improve this answer
11
  • 6
    Note that using the standard logging mechanism like this will not create persistent logs by default. To do that, you'll need to create /var/log/journal, and then run sudo systemctl restart systemd-journald
    – mlissner
    Commented May 8, 2015 at 2:21
  • 1
    what syslog facility and priority?
    – jrwren
    Commented Jul 7, 2015 at 16:31
  • 2
    This worked for me: StandardOutput=syslog+console and StandardError=syslog+console after that all output from my unit appeared in journalctl. The default setting was wrong apparently. (Such as DefaultStandardOutput in /etc/systemd/system.conf)
    – gregn3
    Commented Jul 22, 2015 at 12:23
  • 3
    -f was helpful for me. Followed the log as changes occur (use case was following minecraft server which was running as a daemon)
    – blaughw
    Commented Nov 21, 2016 at 4:55
  • 3
    This is driving me crazy... On a standard Debian stretch journalctl won't show me any of the standard output whatsoever. I even use /usr/bin/stdbuf -oL <cmd> and an explicit StandardOutput=journal. Still nothing.
    – jlh
    Commented Dec 18, 2018 at 9:14
128

Shorter, simpler, non-legacy answer:

sudo journalctl -u [unitfile]

Where [unitfile] is the systemd .service name. Eg, to see messages from myapp.service, 

sudo journalctl --unit=myapp

To follow logs in real time: 

sudo journalctl -f -u myapp
Improve this answer
7
  • 5
    Note that you might have to sudo if you get a No journal files found error.
    – bigjosh
    Commented Nov 17, 2015 at 15:51
  • 11
    syslog is not legacy...
    – mrr
    Commented Jun 26, 2016 at 5:18
  • 4
    It is in current Linux distros. You might really like syslog but that doesn't change what they ship with.
    – mikemaccana
    Commented Jun 26, 2016 at 11:13
  • 3
    Sure. And it also uses syslog. My point was not that systemd is not used, but that syslog is not legacy.
    – labyrinth
    Commented Mar 10, 2017 at 16:46
  • 14
    @JECompton if all logging on current Linux distros uses journald, and syslog is not necessary and only used for compatibility, then it logically follows that syslog is legacy.
    – mikemaccana
    Commented Mar 13, 2017 at 11:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .