0

This is a follow-up question, UNIX-focused

i m trying to implement my own shell and i got stuck in the pipe feature

after some research i found out that i need to run pipe(2) to share data between processes but i don't why i need that my first thought is to change the second process stdin with stdout of the first one my idea

so can anyone explain to me why we need to use pipe(2) like that pipe

Improve this question
4
  • 1
    The difficulty with having a stdout directly connected to a stdin is that both processes need to have issued their respective write and read, for identical transfer sizes, at the same time. Otherwise, there is nowhere to store the data until it can be directly transferred between the processes. What the pipe(2) syscall does is to invoke the kernel to provide the buffering, and the process synchronisation, to enable the two processes enough slack to each run independently. They each get a filedes to their respective ends of that service.
    – Paul_Pedant
    Jun 18 at 20:31
  • @Paul_Pedant, or well, you could say that the difficulty of connecting stdin to stdout directly is that you just can't do it, the system doesn't work like that. Not that I can see why buffering would be a hard requirement for pipes to work. As long as the processes don't depend on each other in a loop, they could always wait for the other to reach a corresponding point when reading or writing. And there's no reason the transfer sizes would need to match exactly: if one writes 300 B and the other reads 100 B, just transfer 100 B and wait for the next read call for the remaining 200 B.
    – ilkkachu
    Jun 19 at 9:28
  • @ilkkachu I may have phrased my comment badly. I am saying that, if A and B are communicating directly through a pair of fds that have not been created with pipe(), then every data transfer demands that A is inside a write syscall at the same moment that B is inside a read syscall: without a pipe, there is no intermediate transfer buffer. That is a crippling synchronisation requirement, and any difference in transfer sizes means each process either waits for another transfer, or gets a short read/write result. Short form is indeed "system doesn't work like that".
    – Paul_Pedant
    Jun 19 at 9:56
  • @Paul_Pedant, I don't see why the lack of buffering there would need to be crippling, at least in the sense of features and functionality. It might affect performance, sure, but even now, the size of the pipe buffer is finite and it's quite possible to end up in a situation where one of the processes needs to wait for the other. It doesn't seem like something that would break the basic functionality of a pipeline. Actually, I'd think that if there's enough data, and one of the parts of the pipeline is faster enough than the other, the end result will be that one always waits for the other.
    – ilkkachu
    Jun 19 at 19:19

2 Answers 2

Reset to default
3

Let's assume the usual situation of a shell running on a terminal. All of stdin, stdout, stderr of the shell are connected to that terminal, and reading from one reads input from the user, and writing to one prints stuff for the user to read.

Now, when the shell launches a command, that one also has all of the fds connected to the terminal. Same for the next command the shell launches. Like so:

+---------------+            +---------------+
|  first cmd    |            |  other cmd    |
|stdin   stdout |            |stdin   stdout |
+---------------+            +---------------+
   |       |                    |       |
   |       |                    |       |
   |       |                    |       |
 [------------------ tty -----------------]

Now, you said:

my first thought is to change the second process stdin with stdout of the first one

but if you were to swap two of the fds above, the result would be no different, all the fds were connected to the tty before, so all would be connected to the tty after any swap.

Now, if you meant detaching one of the fds from the tty side, well, that's not really how it works. Fds are more like pointers to kernel data structures, and an fd pointing to a tty is different from an fd pointing to a pipe or a network socket. Or, to look at it another way, an fd is just the process side of a "connection" between a process and the OS.

So, you need another sort of an fd, one that connects to another kind of thingy on the OS end. For example, a pipe. A pipe still isn't a connection just between the two processes, but two connections between the process(es) and the OS:

+---------------+            +---------------+
|  first cmd    |            |  other cmd    |
|stdin   stdout |            |stdin   stdout |
+---------------+            +---------------+
   |       |                    |       |
   |       |                    |       |
   |     [-w-- pipe driver -----r-]     |
   |                                    |
 [------------- tty driver ---------------]

(Note that I assumed above that the tty fds are all open for reading and writing. That's how it is on the Linux systems I've looked at, so it is actually possible to write to stdin, or duplicate an stdin fd to stdout. If they're opened in read-only / write-only mode, then that doesn't work. AFAIK Linux pipes are always one-directional, and have a distinct read-end and a write-end.)

Improve this answer
2
  • Named pipes (as in mkfifo) are an interesting extension. They do not need a common parent (being opened via a filename), and can deal well with multiple writers (multiple readers are a little harder to work with). They may be a more natural candidate for this use case.
    – Paul_Pedant
    Jun 19 at 9:40
  • 1
    @Paul_Pedant, pipes from pipe(2) can also deal with multiple writers / readers. (Equally badly as named pipes.) I don't know about use-cases, but they mentioned implementing a shell and the pipe feature, and that does seem like something where nameless pipes are fine.
    – ilkkachu
    Jun 19 at 19:14
1

Let's suppose you don't use a pipe. Your first process can write to stdout. Your second process can read from stdin.

It's easy enough to close stdin of the second process, but then what do you open to access the stdout of the first process? (Hint: you can't.)

So you create a pipe with well-known endpoints that can be opened for writing and reading.

Improve this answer
1
  • Minor point: both endpoints are already opened, within a single process. Generally, a child process then needs to be created.
    – Paul_Pedant
    Jun 19 at 9:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.