47

I was trying to run flume on Ubuntu 16.04 as systemd service and have following in /etc/systemd/system/flume-ng.service

[Unit]
Description=Apache Flume

[Service]
ExecStart=/usr/bin/nohup /opt/flume/current/bin/flume-ng agent -c /etc/flume-ng/conf -f /etc/flume-ng/conf/flume.conf --name a1 &
ExecStop=/opt/flume/current/bin/flume-ng agent stop

[Install]
WantedBy=multi-user.target

I tried adding following lines

StandardOutput=/var/log/flume-ng/log1.log
StandardError=/var/log/flume-ng/log2.log   

which didn't work for me. I did run systemctl daemon-reload and systemctl restart flume-ng

anyone know how this works ?

6 Answers 6

70

Redirect

The following works only in systemd v236 and newer:

StandardOutput=file:/var/log/flume-ng/log1.log
StandardError=file:/var/log/flume-ng/log2.log

as documented here.

In case of systemd older than v236, you can use:

ExecStart=/bin/sh -c 'exec /usr/bin/my_binary [arguments] >/var/log/flume-ng/log1.log 2>/var/log/flume-ng/log2.log'

Note that this way the whole log files contents will be overwritten each time service restarts.

Append

If you want to maintain file log between service restarts and just append new logged lines to it:

# Works only in systemd v240 and newer!
# (but backported to e.g. RHEL8)
StandardOutput=append:/var/log/flume-ng/log1.log
StandardError=append:/var/log/flume-ng/log2.log

In case of systemd older than v240, you can use:

ExecStart=/bin/sh -c 'exec /usr/bin/my_binary [arguments] >>/var/log/flume-ng/log1.log 2>>/var/log/flume-ng/log2.log'

exec means that shell program will be substituted with my_binary program after setting up redirections without forking. So there will be no difference from running my_binary directly after ExecStart=.

6
  • 4
    Note: Writing to file was only added in systemd version 236.
    – Dave
    Dec 20, 2018 at 5:54
  • @Dave Why log file name ends with a number ? Jan 3, 2019 at 17:30
  • 6
    the file: option creates a new file the first time and then fails on subsequent restarts of the app for me, i tried append: and it didn't work at all Jan 29, 2019 at 17:07
  • Last solution works fine, although it has to be /bin/sh -c ... (at least on Ubuntu).
    – ToFi
    Oct 8, 2019 at 21:32
  • 1
    This worked perfectly for me with Systemd v237. Dec 13, 2021 at 20:14
26

ExecStart=/usr/bin/nohup …

This is wrong. Remove it. This service is not running in an interactive login session. There is no controlling terminal, or session leader, to send a hangup signal to it in the first place.

ExecStart=… &

This is wrong. Remove it. This is not shell script. & has no special shell-like meaning, and in any case would be the wrong way to start a service.

StandardOutput=/var/log/flume-ng/log1.log
StandardError=/var/log/flume-ng/log2.log

These are wrong. Do not use these. systemd already sends the standard output and error of the service process(es) to its journal, without any such settings in the service unit. You can view it with

journalctl -e -u flume-ng.service

4
  • 73
    What if I wanted to send logs to a different file - is it possible or not? if possible, what is the correct way of doing this?
    – AnthonyK
    Mar 31, 2017 at 2:13
  • What is the location of the log file? I thought the question was about it.
    – ka3ak
    Aug 12, 2020 at 13:56
  • @ka3ak assume the logfile is stored at $logfile
    – Jake
    Sep 26, 2020 at 13:27
  • 6
    how is this selected as right answer? the question is how can we redirect service specific logs to a file that user want. not how to tail it.
    – Vignesh SP
    Jun 29, 2021 at 6:40
16

For logging to file without overwriting (system.d version 240+ required):

StandardOutput=append:/var/log/flume-ng/log1.log
StandardError=append:/var/log/flume-ng/log2.log

or

StandardOutput=append:/var/log/flume-ng/log.log
StandardError=inherit
2
  • 9
    This is unfortunately not working for me. The service still starts but nothing appears in the log file and if deleted it is not recreated. Ununtu 18.04, system.d 237 Apr 9, 2019 at 11:08
  • 2
    @SystemicPlural you're right. This was only added in system.d 240 with a PullRequest for this issue: github.com/systemd/systemd/issues/10875
    – ToFi
    Oct 8, 2019 at 21:01
7

I hope this helps someone. for Ununtu

[Unit]
Description=name-service
After=syslog.target
After=mysql.service
Requires=mysql.service
[Service]
Type=idle
PIDFile=/path_to_PID/****.pid
WorkingDirectory=/path_to_directory
User=root
Group=root
ExecStart=****.py
Restart=always
TimeoutStartSec=1min
RestartSec=1min
KillMode=process
StandardOutput=append:/path_to_log/service.log
StandardError=append:/path_to_log/service_error.log
[Install]
WantedBy=multi-user.target

for CENTOS

[Unit]
Description=name-service
After=syslog.target
After=mysql.service
Requires=mysql.service
[Service]
Type=idle
PIDFile=/path_to_PID/****.pid
WorkingDirectory=/path_to_directory
User=root
Group=root
ExecStart=****.py  >>/path_to_log/log.log
Restart=always
TimeoutStartSec=1min
RestartSec=1min
KillMode=process
[Install]
WantedBy=multi-user.target

"After" and "Requires" of necessity. Attension "User" and "Group"

Another option: add logging inside executed scripts.

2
  • What's your question? You should start a new question instead of posting it as an answer here.
    – ewatt
    Oct 9, 2019 at 8:53
  • 2
    I'm sorry. I have written it wrong. "I hope someone will help"->"I hope this helps someone."
    – kubarik
    Oct 9, 2019 at 13:33
1
[Unit]
Description=ngrokd server

[Service]
Type=forking
User=ngrok
Group=ngrok
ExecStart=/bin/sh -c '/usr/local/ngrok/bin/ngrokd \
    -tlsCrt /usr/local/ngrok/tls/srv-ngrok.cert \
    -tlsKey /usr/local/ngrok/tls/srv-ngrok.key \
    -domain="ioio.cool" \
    -httpAddr=:8480 \
    -httpsAddr=:8433 \
    -tunnelAddr=:8422 >> /var/log/ngrokd.log 2>&1 &'
Restart=always
KillMode=process

[Install]
WantedBy=multi-user.target

You can see man systemd.exec, search keywords StandardOutput, and man systemd.service, serach keywords COMMAND LINES

This syntax is intended to be very similar to shell syntax, but only the meta-characters and expansions described in the following paragraphs are understood. Specifically, redirection using "<", "<<", ">", and ">>", pipes using "|", running programs in the background using "&", and other elements of shell syntax are not supported.

1
  • You can remove the trailing ampersand & from the commandline because this is not an interactive shell. The startup will be forked anyway.
    – Ned64
    Sep 5, 2022 at 10:50
1

If you don't get any output at all you should test your script first.

For example a Python script will buffer the output streams (but only when they are piped, not when you test on the console). So you will not see any log messages until the buffer is flushed.

Specify the -u switch ( python3 -u script) or set the PYTHONUNBUFFERED variable to avoid this:

Environment=PYTHONUNBUFFERED=1
1
  • 1
    Wow, this is very helpful. In a perl script setting $|=1; also unbuffers. Fixed two of my units. Apr 28, 2022 at 19:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .