If think most of us agree that filters and pipes are the very foundation of Unix systems.

Pipes and filters are very powerful tools. Almost all Unix utilities use the standard input and output streams by default to make it possible to use them within pipelines.

It is a convention for utilities within Unix to operate standard input and standard output if no other input/output files have been specified.

grep, as, sed, tr, perl, sort, uniq, bash, cmp, cat and many others are all utilities that follow this convention.

But many programming utilities have abandoned this convention.

Reading input

The most obvious example of this is cc (the C compiler).

If you invoke cc with no arguments you get this message:

ryvnf:~$ cc
cc: fatal error: no input files
compilation terminated.

This is not the only example of this:

ryvnf:~$ yacc
/usr/bin/bison: -y: missing operand
Try '/usr/bin/bison --help' for more information.

Lower-level utilities like as read standard input by default. I wonder why that is.

Writing output

This also applies to output.

cc outputs its executable code into a.out by default. The parser generator yacc outputs its generated parser to y.tab.c.

To me using standard input/output streams by default is advantageous because then you can easily connect various utilities. Like this pipe which compiles a yacc parser to executable code in one go without generating intermediate files like y.tab.c:

yacc parser.y | cc -o parser

My question

Why is it that utilities for programming don't use the standard streams by default as many other Unix utilities do?

What is the motivation for not using standard input streams by default for these utilities?

Note that I am aware that you can get cc to read standard input by using cc -x c -. This works but my question remains why it doesn't do this by default.

  • 1
    I would guess that it's because these things are meant to act on source files, which you usually don't generate programmatically; they don't fit that well into the standard Unix 'filter' metaphor, because both their inputs and their outputs are meant to be objects in and of themselves. – Tom Hunt Nov 30 '15 at 17:42
  • Why should it do that by default? Why shouldn't it read from file by default? What if you have multiple files? Why read file, and not filepaths? How would this be of any use for programs larger than one file? – MatthewRock Nov 30 '15 at 17:43
  • @MatthewRock There are many languages that generate C code as an intermediate language. It would be easier to filter their output through cc if standard input/output was used by default. – wefwefa3 Nov 30 '15 at 17:50
  • Most compilers write their output to object files etc. because that's what most people prefer as a default. They don't want to further process it. – Faheem Mitha Nov 30 '15 at 18:22
  • What are these languages? Is this one file? How would this be easier? Were these languages around when cc was designed? – MatthewRock Nov 30 '15 at 19:15

Pipelines don't work for source code because you can't process input as it comes in. You need the entire file loaded before processing begins. It gets even worse when you need multiple files for compilation (.h files for example). If you were reading from stdin you would need to stream in all of the needed files with some method of specifying file breaks between the files you piped in. The problems just grow from there.

The idea behind the pipeline was that it would be a series of simple tasks. Compiling code is NOT a simple task and so it was never designed to be a part of a pipeline. Also pipeline theory said that all communication between processes in the pipeline should be in plain text to facilitate portability of individual components. By definition the output of cc or yacc or ld or anything else involved in compiling code is binary data which doesn't fit the model.

Not the answer you're looking for? Browse other questions tagged or ask your own question.