0

I recently stumbled over a surprising issue with an invalid /dev/stderr on an up-to-date cygwin which is also present on a well-matured Debian installation. (Edit: Opposed to what I originally thought, my Debian system does not expose this error but simply produces the desired output. I now must suppose that this is a cygwin bug.)

Background: I'm using tools which produce thousands of lines of output (specifically: version control systems on a large production system). I'm running them script-controlled and wanted to optionally redirect the noisy tool output to a log file. A simple solution seemed to always redirect their (stderr and stdout) output to a file system destination which was stored in an environment variable. If output to the terminal (or some user-controlled destination) was desired, the destination DBG_STDERR would simply be "/dev/stderr", otherwise some temp file name. A typical tool execution line would then resemble noisy_command >> "$DBG_STDERR" 2>&1.

This works fine unless I pipe the output of the script. Here is a minimal reproduction:

$ uname -a
CYGWIN_NT-6.1-WOW xxxxxxx 2.8.1(0.312/5/3) 2017-07-03 14:06 i686 Cygwin

$ bash --version
GNU bash, version 4.4.12(3)-release (i686-pc-cygwin)

$ cat say-something.sh
#!/bin/sh
echo something > /dev/stderr

$ (x=$(./say-something.sh 2> /dev/stderr)) 2>&1 |cat
./say-something.sh: line 2: /dev/stderr: No such file or directory

$ (x=$(./say-something.sh 2> /dev/stderr)) 2>&1
something

$ (x=$(./say-something.sh 2> /dev/stderr))  |cat
something

$ x=$(./say-something.sh 2> /dev/stderr) 2>&1 |cat
something

Of course all the redirections and nested shells look funny out of context. The extra shell is necessary because say-something.sh would actually be called by another script. The redundant redirection of fd 2 to stderr is the "switch" to facilitate optional redirection to a file (/dev/stderr, or a different path, is actually the configurable contents of a variable).

It seems as if all constituents of this pipeline are necessary, as the experiments after the failed example show: They all succeed.

  • We need the final pipe of stdout
  • We need the copying of stderr to stdout by the caller
  • We need the outer shell around the command substitution.
4

The name /dev/stderr is actually valid when redirecting to a pipe. What may be not possible is to open the final target of /dev/stderr directly. Just see:

$ (echo Testing testing > /dev/stderr) |& cat
Testing testing

The pipe created by | or |& is usually an anonymous pipe; the name shown does not correspond to an object in the filesystem. For an illustration you can try something simple like:

$ ls -la /dev/fd/ |& cat
total 0
dr-x------ 2 alexp alexp  0 Jul  6 18:23 .
dr-xr-xr-x 9 alexp alexp  0 Jul  6 18:23 ..
lrwx------ 1 alexp alexp 64 Jul  6 18:23 0 -> /dev/pts/4
l-wx------ 1 alexp alexp 64 Jul  6 18:23 1 -> pipe:[1058859]
l-wx------ 1 alexp alexp 64 Jul  6 18:23 2 -> pipe:[1058859]
lr-x------ 1 alexp alexp 64 Jul  6 18:23 3 -> /proc/4335/fd

It is very unusual to try to open the (final) target of /dev/stderr; the name /dev/stderr is provided in order to avoid bothering with finding out the actual target.

  • As an aside, if your script needs to know where standard error is currently going, you can always navel-gaze at /proc/$$/fd/2. – DopeGhoti Jul 6 '17 at 15:36
  • Hm. I have the annoying situation that I cannot reproduce my original error. You are right: A redirection to /dev/stderr works in my test script even if one cannot ls the symlink target. The error I originally had was actually (from my memory) /dev/stderr: No such file or directory, and I thought that the target does not exist in the file system is the root cause. Seems not to be so, so I need to reproduce the error in the original script first. (That error was quite puzzling because of course /dev/stderr is supposed to always exist...) – Peter A. Schneider Jul 6 '17 at 20:21
  • @PeterA.Schneider: /dev/stderr exists unless it doesn't, for example because the process (or one of its ancestors) has closed file descriptor 2... – AlexP Jul 6 '17 at 20:56
  • @AlexP Yes, true. Apart from an explicit close, what else could cause it? (My original case involved nested commands, and some kind of race condition may be possible. I think I had a case when the same command resulted in the /dev/stderr: No such file or directory error in one run but not in the next one. But it is also possible that I piped it to less in the first run but not in the second.) – Peter A. Schneider Jul 7 '17 at 6:56
  • I edited the example, it resembles better what I did originally. The error does not occur on Debian, actually. – Peter A. Schneider Jul 9 '17 at 22:05
1

I think the problem ist the readline-process that is spawned. It gets its own pipe for the redirection which is closed when the process stops (the pid you get is not that of the shell but that of the readlink-process). The pipe gets invalid when process exits. Try to use fifos/named pipes.

  • You have a good point; it's possible that the stderr I'm looking at is not the shell's which is interpreting the script and hence the pipe is closed by the time I try to ls it. – Peter A. Schneider Jul 6 '17 at 20:17
  • Needless to say that in my original use case I never stored the target of a particular /dev/stderr which will be obsolete once the respective process ends (the way I do in my test here), but always used a literal /dev/stderr which should be valid as long as the process exists and does not close it (as AlexP helpfully pointed out). – Peter A. Schneider Jul 7 '17 at 6:59
  • I provided a better example. Storing the path and trying to access it after the process had exited was surely a bug ;-). – Peter A. Schneider Jul 9 '17 at 22:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.