1

When I ssh to a server and start a script that is supposed to run for several days, it stops executing some time after I log off. Why? I suppose the server runs 24/7.

And what does screen/tmux do to prevent this from happening?

4

You need to use the nohup command. A explanation is provided here. Basically it detaches the program you are running from your terminal session it is closed the program does not die with its parent. tmux and screen also detach the programs they run that is why you get the same behavior.

3

From an operating system's point of view, every running program is a process. When the kernel is done initializing it will start one process, mostly init. Whether it is SYSV init or systemd is not relevant for this discussion, one process is started. Every other program is started by another process. This creates a relationship between processes, the starting process (a.k.a. "parent") and the started process (a.k.a. "child"). The kernel is aware of these relations.

When a process exits in Linux/Unix, the kernel sends to all its child processes the signal number 15 (SIGTERM). It can be caught by the process and the process should do whatever it has to do to exit in a safe way. You may know also know the signal number 9 (SIGKILL). This signal cannot be caught by the process: the kernel stops the process by itself.

See this pstree:

init─┬─acpid
     ├─atd
     ├─cron
     ├─dbus-daemon
     ├─dhclient
     ├─dhcpd
     ├─exim4
     ├─6*[getty]
     ├─lwresd───6*[{lwresd}]
     ├─named───6*[{named}]
     ├─nmbd
     ├─portmap
     ├─rpc.statd
     ├─rsyslogd───2*[{rsyslogd}]
     ├─smbd───2*[smbd]
     ├─sshd───sshd───bash───screen───screen───bash───pstree
     └─udevd───2*[udevd]

You can see that pstree is a child process of bash, and bash a child process of screen. When I logout from the machine, the bash after sshd exits and the kernel sends a signal 15 to the child process screen. BUT, screen doesn't react to this. So screen's new parent process is now process number 1 (init). See in this pstree:

init─┬─acpid
     ├─atd
     ├─cron
     ├─dbus-daemon
     ├─dhclient
     ├─dhcpd
     ├─exim4
     ├─6*[getty]
     ├─lwresd───6*[{lwresd}]
     ├─named───6*[{named}]
     ├─nmbd
     ├─portmap
     ├─rpc.statd
     ├─rsyslogd───3*[{rsyslogd}]
     ├─screen───bash
     ├─smbd───2*[smbd]
     ├─sshd───sshd───bash───pstree
     └─udevd───2*[udevd]

In fact, this is what happens when you detach the screen (ctrl+a - d). So all sub-processes of screen keep running after disconnecting or detaching from screen. When you run a process without screen or tmux, it will get a SIGTERM signal.

2
  • 2
    No, the kernel does not send SIGTERM to the children of a process when it dies. It may send SIGHUP under some circumstances (and it has to do with session handling, not parent-child relationship). It can't see what SIGILL has to do with all that. Dec 9 '13 at 21:44
  • Accepting this as it answers why, but would be nice if someone would correct the mistake. Dec 10 '13 at 7:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.