Tag Info

Hot answers tagged

15

Circular I/O Loop Implemented with tail -f This implements a circular I/O loop: $ echo 1 >file $ tail -f file | while read n; do echo $((n+1)); sleep 1; done | tee -a file 2 3 4 5 6 7 [..snip...] This implements the circular input/output loop using the sine algorithm that you mentioned: $ echo 1 >file $ tail -f file | while read n; do echo ...


9

You can use a FIFO for this, created with mkfifo. Note however that its very easy to accidentally create a deadlock. Let me explain that—take your hypothetical "circular" example. You feed a command's output to its input. There are at least two ways this might deadlock: The command has an output buffer. It's partially filled, but hasn't been flushed ...


7

I would agree with you - it probably is a generic problem. Some common utilities have some facilities for handling it, though. nl nl, for example, separates input into logical pages as -delimited by a two character section delimiter. Three occurrences on a line all alone indicate the start of a heading, two the body and one the footer. It replaces any ...


4

One possibility is to do this with the vim text editor. It can pipe arbitrary sections through shell commands. One way to do this is by line numbers, using :4,6!nl. This ex command will run nl on the lines 4-6 inclusive, achieving what you want on your example input. Another, more interactive way is to select the appropriate lines using line-selection mode ...


4

Yes, it slows things down. And it basically does have a queue of unwritten data, though that's actually maintained by the kernel—all programs have that, unless they explicitly request otherwise. For example, here is a trivial pipe using pv, which is nice because it displays transfer rate: $ pv -s 50g -S -pteba /dev/zero | cat > /dev/null 50GiB ...


4

There exists several variants of a watch command, some that spawn a shell to interpret a command line made of the concatenation of the arguments passed to watch (with space characters in between). In those you can do: watch 'ls | shuf' same as: watch ls '|' shuf (those watch actually run: "/bin/sh", ["sh", "-c", "ls | shuf"] and are quite dangerous in ...


3

You know, I'm not convinced you necessarily need a repetitive feedback loop as your diagrams portray, so much as maybe you could use a persistent pipeline between coprocesses. Then again, it may be there isn't too much of a difference - once you open a line on a coprocess you can implement typical style loops just writing information to and reading ...


3

In general I would use a Makefile (command make) and try to map your diagram to makefile rules. f1 f2 : f0 command < f0 > f1 2>f2 To have repetitive/cyclic commands, we need to define a iteration policy. With: SHELL=/bin/bash a.out : accumulator cat accumulator <(date) > a.out cp a.out accumulator accumulator: touch ...


3

The processes are started at the same time, and will run concurrently, but they don't have to stop at the same time. The shell will consider the entire pipeline to have terminated (and display a new prompt) when both processes have terminated. If command2 ends before command1 does (or closes its standard input stream), and command1 then attempts to write ...


3

The redirection to file.txt at the end of your paste command is truncating your file before paste has a chance to read it. Try echo 2 | paste file.txt - > file2.txt or if you have sponge installed echo 2 | paste file.txt - | sponge file.txt


3

Many grep implementation will use line buffered when standard output is terminal. When standard output is terminal, it's often an interactive session, you want to get data as soon as possible. So grep will write data to standard output as soon as seeing a newline. When standard output isn't terminal (often meaning non-interactive session), grep will use ...


2

The terms you need in order to use some google-fu are 'redirection' which is what you're doing in process < input.txt and 'piping' which is what you're doing in cat input.txt | process Once you know this, you can find articles like this one http://askubuntu.com/questions/172982/what-is-the-difference-between-redirection-and-pipe which answers your ...


2

If your goal is to send the entire code block to a single process instance then you could accumulate the lines and delay piping until you reach the end of the code block: #!/bin/bash acc="" while read line do if [[ $line == @@inline-code-start* ]] then active=true acc="" elif [[ $line == @@inline-code-end* ]] then active=false # ...


2

You can use -I to define a place-holder which will be replaced with each value of the arguments fed to xargs. For example, ls -1 | xargs -I '{}' echo '{}' will call echo once per line from ls's output. You'll often see '{}' used, presumably because it's the same as find's place-holder. In your case, you also need to pre-process file's output to extract ...


2

You don't say what shell are you using. From the behaviour you are describing it's likely zsh. If you have a look in its man page you would notice how redirections are handled. Note that a pipe is an implicit redirection; thus cat bar | sort <foo is equivalent to cat bar foo | sort (note the order of the inputs). Otherwise, regular ...


2

Your understanding is not quite correct. In a | b the stdout output of process a connected through a pipe to stdin of process b. The problem with your code is that with an additional redirection from somefile to process b you will use two different methods at the same time to connect to stdin of process b. Don't do that! The question is; what do you try to ...


2

Edit added an option to define a user-provided filter #!/usr/bin/perl -s use IPC::Open2; our $p; $p = "nl" unless $p; ## default filter $/ = "\@\@inline-code-end\n"; while(<>) { chomp; s/\@\@inline-code-start\n(.*)/pipeit($1,$p)/se; print; } sub pipeit{my($text,$pipe)=@_; open2(my $R, my $W,$pipe) || die("can open2"); local $/ = ...


2

The simplest fix I can think of is to not use nl but count the lines yourself: #!/usr/bin/env bash while read line do if [[ $line == @@inline-code-start* ]] then active=true elif [[ $line == @@inline-code-end* ]] then active=false elif [[ $active = true ]] then ## Count the line number let num++; ...


2

From write(2), Errors       ⋮ EPIPE fd is connected to a pipe or socket whose reading end is closed.  When this happens the writing process will also receive a SIGPIPE signal.  … In other words, pipes and sockets, while not the same thing, are similar enough that Unix uses the same mechanism to inform a writing process that there is ...


1

In a command where we have lot of pipes redirecting their outputs further, is there any way to get to know the value of echo $? till half of it's execution or till certain number of pipes, as opposed to the overall status of the whole command's? In bash, there's a PIPESTATUS variable, which is an array containing the exit status of each command in the ...


1

You have this command ls -lrt --time-style=+"%b %d %Y %H:%M:%S" /some/path/*.dat|head -1|tr -s " "|cut -d " " -f 9|date --date - +%s The problem with this is that date does not expect its parameters via stdin, so you need to split it into two parts. You've already attempted this in the second part of your Question: var1=$(ls -lrt --time-style=+"%b %d %Y ...


1

why not use a here document like in the old days ? #!/bin/bash export TMPDIR=`mktemp -d /tmp/selfextract.XXXXXX` ## other stuff base64 -d <<EOF | tar xvf -C $TMPDIR ## base64 encoded EOF you produce base64 encoded this way tar cf - my_dir | base64 > /some/place (including it int self extract archive is left to the reader)


1

That's a job for awk. #!/usr/bin/awk -f $0 == "@@inline-code-start" {pipe = 1; next} $0 == "@@inline-code-end" {pipe = 0; close("nl"); next} pipe {print | "nl"} !pipe {print} When the script sees the start marker, it notes that it should start piping into nl. When the pipe variable is true (nonzero), the output is piped into the nl command; when the ...



Only top voted, non community-wiki answers of a minimum length are eligible