How can a service with PrivateTmp=true access a unix socket in the /tmp directory (e.g. to submit Torque jobs from PHP running in Apache)

Question

We have a webserver that performs scientific calculations submitted by users. The calculations can be long-running, so we use The Torque resource manager (aka pbs_server) to distribute/schedule them on a handful of compute nodes. Torque makes use of a unix domain socket in the /tmp directory for communication but the http server (and processes forked from it) can't access the true /tmp directory, so to those processes, the socket appears to be missing, resulting in an error.

The Details:

• The webserver is running Apache, which runs as a service with the systemd property PrivateTmp=true set. This casuses the service to have its own /tmp directory unrelated to the "true" root /tmp.
• The jobs are actually submitted from PHP (running in the Apache process). PHP makes a system call to qsub, which is a Torque command to submit a job. Because qsub is called from PHP, it inherits the "fake" /tmp directory from Apache.
• qsub internally attempts to connect to the unix socket located at /tmp/trqauthd-unix. But since it doesn't see the real /tmp directory, it fails with the following error: Error in connection to trqauthd (15137)-[could not connect to unix socket /tmp/trqauthd-unix: 2]

The only solution I could achieve was to edit the httpd.service file under systemd and change PrivateTmp to false. This DID fix the problem. But I'd rather not do this because (I assume) PrivateTmp was set to true for good reason.

What I want to know is whether there is any way to have the socket created in a different location or to somehow make a link to the socket that could be used from within Apache (and its forked processes).

Creating a link to the socket is trivial, but it doesn't solve the problem because I don't know of any way to configure qsub to look for the socket in a different location.

Note that the socket is created by the trqauthd service (a Torque program that performs user authorization for running jobs). The documentation for trqauthd mentions (in an obscure note) that the location of the socket can be configured, but there is no indication in any of the documentation about how that can be achieved (and more importantly, how to let qsub and other commands know about the new location).

Thanks for any suggestions that might help me find a way to submit jobs to Torque from PHP without disabling PrivateTmp for Apache.

Answer 1

In 2013, trqauthd stopped using IP sockets and switched to a Unix domain socket in the server's home directory.

Later that same year, trqauthd switched from the home directory to /tmp.

As you can see, the only option that Adaptive Computing has given to you for altering /tmp/trqauthd-unix is to re-compile the programs from source, changing the --with-trqauthd-sock-dir build configuration option to denote somewhere other than /tmp. (/run/trqauthd perhaps?)