23

On occasion process substitution will not work as expected. Here is an example:

Input:

gcc <(echo 'int main(){return 0;}')

Output:

/dev/fd/63: file not recognized: Illegal seek
collect2: error: ld returned 1 exit status

Input:

But it works as expected when used with a different command:

grep main <(echo 'int main(){return 0;}')

Output:

int main(){return 0;}

I have noticed similar failures with other commands (i.e. the command expecting the file from the process substitution can't use /dev/fd/63 or similar). This failure with gcc is just the most recent. Is there some general rule that I should be aware of to determine when process substitution will fail in this way and should not be used?

I am using this BASH version on Ubuntu 12.04 (I've also seen this in arch and debian):
GNU bash, version 4.3.11(1)-release (i686-pc-linux-gnu)

  • 1
    illegal seek looks like the answer - the |pipe that bash points the executed program to is not a seekable file. probably if you cannot successfully echo data | command /dev/fd/0 at a program then you'll have similar luck w/ <(cmd). It doesn't provide an on-disk file - it just substitutes an argument that points to a pipe file descriptor. – mikeserv Oct 25 '14 at 0:15
  • 2
    In this particular case, although gcc can accept standard input, it (by default) uses the filename extension to determine the language. So try gcc -xc <(echo 'int main(){return 0;}') (which sets the language to C explicitly). – steeldriver Oct 25 '14 at 0:16
  • I was directed here in response to my own question, which seems likely to be another example of this. superuser.com/questions/1243405. Thanks for phrasing the question better than I was able. – Jonathan Hartley Aug 23 '17 at 17:01
27

Process substitution results in a special file (like /dev/fd/63 in your example) that behaves like the read end of a named pipe. This file can be opened and read, but not written, not seeked.

Commands that treat their arguments as pure streams work while commands that expect to seek in files they are given (or write to them) won't work. The kind of command that will work is what is usually considered a filter: cat, grep, sed, gzip, awk, etc... An example of a command that won't work is an editor like vi or a file operation like mv.

gcc wants to be able to perform random access on its input files to detect what language they are written in. If you instead give gcc a hint about the input file's language, it's happy to stream the file:

gcc -x c <(echo 'int main(){return 0;}')

The simpler more straightforward form without process substitution also works:

echo 'int main(){return 0;}' | gcc -x c -

Note that this is not specific to bash. All shells that support process substitution behave the same way.

  • +1 for the gcc workaround but I'm not sure about your point concerning files. The <() format should act like a file for all intents and purposes. In fact, I don't know of any commands that expect a file that won't be happy with <(). The ones that don't work are those that expect file names, not files. For example, grep -f expects a file and works fine with <(). – terdon Oct 25 '14 at 12:52
  • 4
    @terndon For sure <() produces a file name (the construct expands to /proc/self/fd/something on my system). This name, when opened, acts like the read end of a named pipe (S_IFIFO) rather than a regular file (S_IFREG) in that is supports read() and others but not seek(). – Celada Oct 25 '14 at 18:16
  • 7
    Note that zsh supports a 3rd form of process substitution that uses temporary files especially for that purpose: gcc =(echo 'int main(){return 0;}') – Stéphane Chazelas Dec 2 '14 at 17:15
  • maybe related, but works with <(echo '...') but not with <(git show ...). any idea why that could be? – Jörn Hees Jun 14 '15 at 20:23
  • 2
    GCC does not "perform random access on its input files to detect what language they are written in." It just looks at the filename's extension. If the filename does not have an extension (or if it has one which isn't recognized), GCC assumes the file is an object file or linker script and passes it to ld (which does detect object formats). -x is not a hint; it is a declaration. If you specify -x f95, GCC will pass the file to the Fortran-95 compiler regardless of its name or contents. See gcc.gnu.org/onlinedocs/gcc-8.1.0/gcc/Overall-Options.html – rici Jun 12 '18 at 4:33

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.