The /tmp/in, /tmp/out and /tmp/err are named pipes, already created and opened by some process (for reading, writing and writing, respectively).
I would like to create a new process that pipes its stdin into /tmp/in, and writes the contents of /tmp/out to its stdout, and the contents of /tmp/err to its stderr as they become available. Everything should work in a line buffered fashion. The process should exit when the other process, that created /tmp/in, stops reading and closes /tmp/in. The solution should work on Ubuntu, preferably without installing any extra package. I would like to solve it in a bash script.
mikeserv pointed out that without an SSCCE, it is hard to understand what I want. So, below is an SSCCE, but keep in mind that it is a minimal example so it is pretty silly.
The original setup
A parent process launches a child process and communicates with it through the child's stdin and stdout line-by-line. If I run it, I get:
$ python parent.py
Parent writes to child: a
Response from the child: A
Parent writes to child: b
Response from the child: B
Parent writes to child: c
Response from the child: C
Parent writes to child: d
Response from the child: D
Parent writes to child: e
Response from the child: E
Waiting for the child to terminate...
Done!
$
parent.py
from __future__ import print_function
from subprocess import Popen, PIPE
import os
child = Popen('./child.py', stdin=PIPE, stdout=PIPE)
child_stdin = os.fdopen(os.dup(child.stdin.fileno()), 'w')
child_stdout = os.fdopen(os.dup(child.stdout.fileno()))
for letter in 'abcde':
print('Parent writes to child: ', letter)
child_stdin.write(letter+'\n')
child_stdin.flush()
response = child_stdout.readline()
print('Response from the child:', response)
assert response.rstrip() == letter.upper(), 'Wrong response'
child_stdin.write('quit\n')
child_stdin.flush()
print('Waiting for the child to terminate...')
child.wait()
print('Done!')
child.py, must be executable!
#!/usr/bin/env python
from __future__ import print_function
from sys import stdin, stdout
while True:
line = stdin.readline()
if line == 'quit\n':
quit()
stdout.write(line.upper())
stdout.flush()
The desired setup and a hackish solution
Neither the parent's source file nor the child's source file can be edited; it is not allowed.
I rename the child.py to child_original.py (and make it executable). Then, I put a bash script (a proxy or a middle man if you wish) called child.py, start the child_original.py myself before running python parent.py and have the parent.py call the fake child.py which is now my bash script, forwarding between the parent.py and the child_original.py.
The fake child.py
#!/bin/bash
parent=$$
cat std_out &
(head -n 1 shutdown; kill -9 $parent) &
cat >>std_in
The start_child.sh to start child_original.py before executing the parent:
#!/bin/bash
rm -f std_in std_out shutdown
mkfifo std_in std_out shutdown
./child_original.py <std_in >std_out
echo >shutdown
sleep 1s
rm -f std_in std_out shutdown
The way of executing them:
$ ./start_child.sh &
[1] 7503
$ python parent.py
Parent writes to child: a
Response from the child: A
Parent writes to child: b
Response from the child: B
Parent writes to child: c
Response from the child: C
Parent writes to child: d
Response from the child: D
Parent writes to child: e
Response from the child: E
Waiting for the child to terminate...
Done!
$ echo
[1]+ Done ./start_child.sh
$
This hackish solution works. As far as I know, it does not meet the line buffered requirement and there is an extra shutdown fifo to inform the start_child.sh that child_original.py has closed the pipes and start_child.sh can safely exit.
The question asks for an improved fake child.py bash script, meeting the requirements (line buffered, exits when the child_original.py closes any of the pipes, does not require an extra shutdown pipe).
Stuff I wish I had known:
- If a high-level API is used for opening a fifo as a file, it must be opened for both reading and writing, otherwise the call to
openalready blocks. This is incredibly counter-intuitive. See also Why does a read-only open of a named pipe block? - In reality, my parent process is a Java application. If you work with an external process from Java, read the stdout and stderr of the external process from daemon threads (call
setDamon(true)on those threads before starting them). Otherwise, the JVM will hang forever, even if everybody is done. Although unrelated to the question, other pitfalls include: Navigate yourself around pitfalls related to the Runtime.exec() method. - Apparently, unbuffered means buffered, but we don't wait until the buffer gets full but flush it as soon as we can.
