Who cuts off first symbol of process substituted file and for what reason?

Question

$ type 1.sh
#!/bin/bash -eu
php <(echo 12)
$ ./1.sh
2

$ type 2.sh
#!/bin/bash -eu
cat <(echo 12)
$ ./2.sh
12

$ type 3.sh
#!/bin/bash -eu
echo 12 | php
$ ./3.sh
12

$ type 4.sh
#!/bin/bash -eu
rm -f named_pipe
mknod named_pipe p
echo 12 > named_pipe
$ ./4.sh
$ php named_pipe   # from another window
2

I tested it on Debian (php-5.4.14, bash-4.1.5) and Arch Linux (php-5.4.12, bash-4.2.42).

Answer 1

It's PHP, of course. Pipes wouldn't eat the first character of the file. PHP is reading all the characters, but it isn't outputting the first one. So far you cannot tell whether the problem is in the input or in the output: it could be that PHP is not outputting the first character for some reason.

A little experiment shows that the problem is indeed with the input.

$ php <(echo '<?php echo "hello" ?>') 
?php echo "hello" ?>
$ php <(echo ' <?php echo "hello" ?>')
hello$ 

PHP is eating up the first character of the script, only when the script is given by a file name (not when there is no command line argument and the script is read from standard input), only when the script file is a pipe or other non-seekable file (not when the script file is seekable, e.g. when it's a regular file).

What's going on is that at the very beginning, before the normal PHP parser kicks in, the command line processor checks whether the script begins with a shebang line. If the script begins with the two characters #!, PHP skips the first line. This way you can write a PHP script like this

#!/usr/bin/php
first line
<?php echo "second line"?>

and that script will output

first line
second line

and no spurious #!/usr/bin/php at the beginning.

The shebang detector works this way:

• Read the first character.
• If the first character is #, read another character.
• If the first two characters are #!, keep reading until the first newline character.
• If the first two characters are not #!, rewind to the beginning of the file.
• Start normal PHP parsing.

If the script file is non-seekable, the rewind step fails, but PHP doesn't attempt to detect that, so the first character of the file is lost (and also the second character if the first was a #). This is a bug in the PHP command line interpreter.

You can see the code for yourself (in the function cli_seek_file_begin).

Answer 2

I can reproduce what you're seeing on Ubuntu:

#!/bin/bash -eu
rm -rf named_pipe
mkfifo named_pipe
echo 12 >  named_pipe


$ ./namedpipe.sh &  # background it
$ php named_pipe    # same terminal; don't need another window
2

It's something with php. I could not reproduce it with cat (which would show that the fifo code in the kernel is seriously broken).

An strace log over php shows that it has read the data from the pipe: two digits and a newline:

open("named_pipe", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFIFO|0664, st_size=0, ...}) = 0
mmap2(NULL, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb6fdc000
read(3, "12\n", 65536)                  = 3

The next thing it tries of file descriptor 3 is an lseek:

_llseek(3, 0, 0xbf93c640, SEEK_SET)     = -1 ESPIPE (Illegal seek)

This is where things may be going wrong. Suppose that this is called through some buffered I/O library which gets confused by pipes and messes up its buffer.

At this point, numerous other operations take place.

Later it tries a few more operations, including an attempt to get tty settings and one more read which clearly indicates EOF:

ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbf93a2e8) = -1 EINVAL (Invalid argument)
fstat64(3, {st_mode=S_IFIFO|0664, st_size=0, ...}) = 0
read(3, "", 65536)                      = 0

PHP doesn't believe that a zero return on a blocking read of a Unix pipe means EOF and so it tries again:

read(3, "", 65536)                      = 0
close(3)              

The very next two lines in the trace are these. The very buffer that was allocated prior to that read is released (perhaps the stream buffer?) Followed by the chopped output:

munmap(0xb6fdc000, 65536)               = 0
write(1, "2\n", 2)                      = 2

So that pretty much answers the question of what software is responsible. If you want to dig into it more, a way forward would be to get a debug build of php and step into it with gdb.

---

Addendum:

The behavior is quite different when php reads from standard input, like in the echo 12 | php case. For example, the llseek operation is never tried on file descriptor 0. Also, the stream is never closed (of course) and there are no intervening operations between the read and write. The following appears as a contiguous block:

read(0, "12\n", 4096)                   = 3
read(0, "", 4096)                       = 0
read(0, "", 4096)                       = 0
write(1, "12\n", 3)                     = 3

One would expect that opening a file named on the command line goes through different flow compared to reading from standard input (probably represented by some specially set up global standard input stream object).