3

So far, I know the piping mechanism as a way to connect a series of commands by connecting the stdout of one command to the stdin of the next command until the last command is reached, which connects its stdout with the display or a file.

Would it be possible, however, to make a loop out of commands, so the last command's stdout connects to the first command's stdin and maybe by using tee somehow the changing values of a certain output could be displayed?

  • 4
    I edited your post with a personal opinion; feel free to revert it, of course, but I'm just trying to keep the question clear and ... non-revolting. – Jeff Schaller May 27 '17 at 20:13
  • if I made a calvinistic protestant happy by improving the world by making it more like god imagined it with the power of the holy editor function, I am more than pleased – sharkant May 27 '17 at 20:18
  • 2
    Do you have a use case for where something like this would be a solution? Or is it a purely intellectual exercise? – Kusalananda May 27 '17 at 20:33
  • i am bloody beginner, thus I do not know enough for making reasonable solutions for anything, so like chanting A B C i am bound to intellectual exercises and speculations, hoping not asking too stupid questions or becomming notorious for contributing waste without benefit – sharkant May 27 '17 at 20:39
  • 1
    @JeffSchaller I was like "what could Jeff possibly mean, revolting" and then i clicked the edit history – cat May 28 '17 at 1:57
1

Well you certainly could by just make a loop and use variables:

while true; do
    a=$(echo "$a" | grep "Hey" | cut -d" " -f2 | tee -a log)
done

That would save the last output which would be used at the beginning again

  • a = command1|command2|command3 so this structure would save the std output of command3 into variable a, but what if command3 does not have one but a series of values? – sharkant May 27 '17 at 20:49
  • 1
    It would all be saved as one whole string in the Variable a and be used again in command 1 – ADDB May 27 '17 at 20:51
  • 2
    I'm not sure if this answers the actual question: that's a whole different kind of a loop. – ilkkachu May 28 '17 at 8:30
  • Nah it doesn't, he accepted it just for fun. He asked how do built a loop that's using the stdout as stdin, and that is what this loop does – ADDB May 28 '17 at 8:32
7

I'm not sure about all the shells there are, but in Bash it's possible, though not with unnamed pipes. So not with the | symbol. But if you create a named pipe:

mkfifo fifo

Then you can use it:

<fifo cat | cat >fifo &

Now the pipeline works in the background, but does nothing. But if you feed the pipe from outside the pipeline:

echo x >fifo

The pipeline will unblock and go on forever. Or until you drain the pipe:

cat fifo

The output will appear once:

x

To make this a little sophisticated, the pipeline may be this:

<fifo cat | xargs -I@ echo @x >fifo &

So it will add an x to the output at each iteration. Of course it will, but only once the iterations start, meaning as soon as the pipe unblock, that is as soon as there's something to read. As previously, this can be started by hand:

echo x >fifo

And now take a look at what top shows. There should be quite a lot of activity of both cat and xargs.

And the same as before, if you drain the pipeline, you should see a lot of xs in the terminal, and the pipeline will block.

It would be a valid question, why does the pipeline get drained. Why is the cat command committed in the terminal leaving nothing in the circuit. I don't know this.

  • 3
    This is heavily dependent on the size of the standard IO buffer size. If any data between two parts of the pipeline exceeds the buffer size, the pipeline would block. – Kusalananda May 27 '17 at 20:53
  • @Kusalananda That will be the limit. And it may be an answer why the pipeline gets drained. – Tomasz May 27 '17 at 20:55
2

No, that's not possible. At least not without some serious file descriptor trickery, or a loop as ADDB is showing in his answer.

There are not enough existing real life problems to warrant implementing the syntax for a self-feeding circular pipeline, and I can see issues with determining when the pipeline would be considered "finished", what its exit status would be, and how to ensure that it's not blocking.

The only type of applications I can think of at the moment are in the field of numerical analysis, or a few other computational fields, where a numerical solution is refined using an iterative method until an error estimate is reduced below a certain point.

Those types of problems require floating point arithmetics (often at a high precision), which is not supported by most shells. These problems also generally require a faster execution environment than what a shell supplies.

  • what do you mean by "execution environment" ? – sharkant May 27 '17 at 20:52
  • 1
    @sharkant In this case, I mean that problems of the character that I mentioned are usually programmed to run in parallel on a cluster of computational nodes, using some form of network protocol for communication. – Kusalananda May 27 '17 at 20:56
  • Well, if you make loop of processes connected with pipes, the shell doesn't come into it while the processes run. Though shared memory might still be a better way to communicate between the processes. – ilkkachu May 28 '17 at 8:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.