15

On the man page, it just says:

-m Job control is enabled.

But what does this actually mean?

I came across this command in a SO question, I have the same problem as OP, which is "fabric cannot start tomcat". And set -m solved this. The OP explained a little, but I don't quite understand:

The issue was in background tasks as they will be killed when the command ends.

The solution is simple: just add "set -m;" prefix before command.

9

Quoting the bash documentation (from man bash):

JOB CONTROL
       Job  control  refers to  the  ability  to selectively  stop
       (suspend) the execution of  processes and continue (resume)
       their execution at a later point.  A user typically employs
       this facility via an interactive interface supplied jointly
       by the operating system kernel's terminal driver and bash.

So, quite simply said, having set -m (the default for interactive shells) allows one to use built-ins such as fg and bg, which would be disabled under set +m (the default for non-interactive shells).

It's not obvious to me what the connection is between job control and killing background processes on exit, however, but I can confirm that there is one: running set -m; (sleep 10 ; touch control-on) & will create the file if one quits the shell right after typing that command, but set +m; (sleep 10 ; touch control-off) & will not.

I think the answer lies in the rest of the documentation for set -m:

-m      Monitor  mode. [...]                     Background pro‐
        cesses run in a separate process group and a  line  con‐
        taining  their exit status is printed upon their comple‐
        tion.

This means that background jobs started under set +m are not actual "background processes" ("Background processes are those whose process group ID differs from the terminal's"): they share the same process group ID as the shell that started them, rather than having their own process group like proper background processes. This explains the behavior observed when the shell quits before some of its background jobs: if I understand correctly, when quitting, a signal is sent to the processes in the same process group as the shell (thus killing background jobs started under set +m), but not to those of other process groups (thus leaving alone true background processes started under set -m).

So, in your case, the startup.sh script presumably starts a background job. When this script is run non-interactively, such as over SSH as in the question you linked to, job control is disabled, the "background" job shares the process group of the remote shell, and is thus killed as soon that shell exits. Conversely, by enabling job control in that shell, the background job acquires its own process group, and isn't killed when its parent shell exits.

  • Thx for your answer, but how is tomcat/bin/startup.sh related to fg/bg? – laike9m Apr 16 '15 at 15:44
  • 1
    It's not directly related; I was trying to answer "Job control is enabled / what does this actually mean?" in a general way. Perhaps I haven't made it clear, but your script appears to start a background job, which is what the rest of my answer applies to. – dhag Apr 16 '15 at 16:05
1

I'v found this at github issue list, and I think this really answer your question.

It's not really a SSH problem, it's more the subtle behaviour around BASH non-interactive/interactive modes and signal propagation to process groups.

Following is based on https://stackoverflow.com/questions/14679178/why-does-ssh-wait-for-my-subshells-without-t-and-kill-them-with-t/14866774#14866774 and http://www.itp.uzh.ch/~dpotter/howto/daemonize, with some assumptions not fully validated, but tests about how this works seem to confirm.

pty/tty = false

The bash shell launched connects to the stdout/stderr/stdin of the started process and is kept running until there is nothing attached to the sockets and it's children have exited. A good deamon process will ensure it doesn't wait for it's children to exit, fork a child process and then exit. When in this mode no SIGHUP will be sent to the child process by SSH. I believe this will work correctly for most scripts executing a process that handles deamonizing itself and doesn't need to be backgrounded. Where init scripts use '&' to background a process then it's likely that the main problem will be whether the backgrounded process ever attempts to read from stdin since that will trigger a SIGHUP if the session has been terminated.

pty/tty = true*

If the init script backgrounds the process started, the parent BASH shell will return an exit code to the SSH connection, which will in turn look to exit immediately since it isn't waiting on a child process to terminate and isn't blocked on stdout/stderr/stdin. This will cause a SIGHUP to be sent to the parent bash shell process group, which since job control is disabled in non-interactive mode in bash, will include the child processes just launched. Where a daemon process explicitly starts a new process session when forking or in the forked process then it or it's children won't receive the SIGHUP from the BASH parent process exiting. Note this is different from suspended jobs which will see a SIGTERM. I suspect the problems around this only working sometimes has to do with a slight race condition. If you look at the standard approach to deamonizing - http://www.itp.uzh.ch/~dpotter/howto/daemonize, you'll see that in the code the new session is created by the forked process which may not be run before the parent exits, thus resulting the random sucess/failure behaviour mentioned above. A sleep statement will allow enough time for the forked process to have created a new session, which is why it works for some cases.

pty/tty = true and job control is explicitly enabled in bash

SSH won't connect to the stdout/stderr/stdin of the bash shell or any launched child processes, which will mean it will exit as soon as the parent bash shell started finished executing the requested commands. In this case, with job control explicitly enabled, any processes launched by the bash shell with '&' to background them will be placed into a separate session immediately and will not receive the SIGHUP signal when the the parent process to the BASH session exits (SSH connection in this case).

What's needed to fix

I think the solutions just need to be explicitly mentioned in the run/sudo operations documentation as a special case when working with background processes/services. Basically either use 'pty=false', or where that is not possible, explicitly enable job control as the first command, and the behaviour will be correct.

From https://github.com/fabric/fabric/issues/395

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.