/tmp/err are named pipes, already created and opened by some process (for reading, writing and writing, respectively).
I would like to create a new process that pipes its stdin into
/tmp/in, and writes the contents of
/tmp/out to its stdout, and the contents of
/tmp/err to its stderr as they become available. Everything should work in a line buffered fashion. The process should exit when the other process, that created
/tmp/in, stops reading and closes
/tmp/in. The solution should work on Ubuntu, preferably without installing any extra package. I would like to solve it in a bash script.
The original setup
A parent process launches a child process and communicates with it through the child's stdin and stdout line-by-line. If I run it, I get:
$ python parent.py Parent writes to child: a Response from the child: A Parent writes to child: b Response from the child: B Parent writes to child: c Response from the child: C Parent writes to child: d Response from the child: D Parent writes to child: e Response from the child: E Waiting for the child to terminate... Done! $
from __future__ import print_function from subprocess import Popen, PIPE import os child = Popen('./child.py', stdin=PIPE, stdout=PIPE) child_stdin = os.fdopen(os.dup(child.stdin.fileno()), 'w') child_stdout = os.fdopen(os.dup(child.stdout.fileno())) for letter in 'abcde': print('Parent writes to child: ', letter) child_stdin.write(letter+'\n') child_stdin.flush() response = child_stdout.readline() print('Response from the child:', response) assert response.rstrip() == letter.upper(), 'Wrong response' child_stdin.write('quit\n') child_stdin.flush() print('Waiting for the child to terminate...') child.wait() print('Done!')
child.py, must be executable!
#!/usr/bin/env python from __future__ import print_function from sys import stdin, stdout while True: line = stdin.readline() if line == 'quit\n': quit() stdout.write(line.upper()) stdout.flush()
The desired setup and a hackish solution
Neither the parent's source file nor the child's source file can be edited; it is not allowed.
I rename the child.py to child_original.py (and make it executable). Then, I put a bash script (a proxy or a middle man if you wish) called child.py, start the child_original.py myself before running
python parent.py and have the parent.py call the fake child.py which is now my bash script, forwarding between the parent.py and the child_original.py.
The fake child.py
#!/bin/bash parent=$$ cat std_out & (head -n 1 shutdown; kill -9 $parent) & cat >>std_in
start_child.sh to start child_original.py before executing the parent:
#!/bin/bash rm -f std_in std_out shutdown mkfifo std_in std_out shutdown ./child_original.py <std_in >std_out echo >shutdown sleep 1s rm -f std_in std_out shutdown
The way of executing them:
$ ./start_child.sh &  7503 $ python parent.py Parent writes to child: a Response from the child: A Parent writes to child: b Response from the child: B Parent writes to child: c Response from the child: C Parent writes to child: d Response from the child: D Parent writes to child: e Response from the child: E Waiting for the child to terminate... Done! $ echo + Done ./start_child.sh $
This hackish solution works. As far as I know, it does not meet the line buffered requirement and there is an extra shutdown fifo to inform the start_child.sh that child_original.py has closed the pipes and start_child.sh can safely exit.
The question asks for an improved fake child.py bash script, meeting the requirements (line buffered, exits when the child_original.py closes any of the pipes, does not require an extra shutdown pipe).
Stuff I wish I had known:
- If a high-level API is used for opening a fifo as a file, it must be opened for both reading and writing, otherwise the call to
openalready blocks. This is incredibly counter-intuitive. See also Why does a read-only open of a named pipe block?
- In reality, my parent process is a Java application. If you work with an external process from Java, read the stdout and stderr of the external process from daemon threads (call
setDamon(true)on those threads before starting them). Otherwise, the JVM will hang forever, even if everybody is done. Although unrelated to the question, other pitfalls include: Navigate yourself around pitfalls related to the Runtime.exec() method.
- Apparently, unbuffered means buffered, but we don't wait until the buffer gets full but flush it as soon as we can.