Questions about FIFO - first-in first-out special file, also known named pipe

learn more… | top users | synonyms (1)

0
votes
0answers
33 views

How to do multi threading using mkfifo? [on hold]

I have a file which having milions of records like below. 8350000014|1234567|NULL|NULL 7336004920|478637|NULL|NULL 9876006473|478632|NULL|NULL 6730000012|1234567|NULL|NULL ...
0
votes
0answers
11 views

Fedora 23: PCH FIFO A underrun error during boot

I managed to install Fedora 23 on my Lenovo W530 Laptop with NVIDIA Quadro K2000M. Everything works absolutely fine, even the external screen which works only when the NVIDIA driver is properly ...
8
votes
1answer
70 views

Why doesn't file redirection to and from a named pipe work, but piping to cat does?

This is a simple echo server in Unix, using nc: mkfifo fifo cat fifo | nc -k -l 4458 -v | cat >fifo (based on this) As I can see it, the data flow works as follows: fifo (my named pipe) | | ...
1
vote
1answer
54 views

Using Named Pipes in Bash created in a C Program

I have a C code that takes a rather large input. At every iteration, it processes a small chunk of it and it should send that processed chunk to S3 using curl. In the end, it will generate a number of ...
3
votes
1answer
44 views

mkfifo pipes, file descriptors and ImageMagick (convert)

I'm in trouble using named pipes (i.e. created with mkfifo) and file descriptors to redirect outputs and grab inputs to/from them within ImageMagick (convert). Basically, with the following bash code ...
2
votes
1answer
40 views

redirect to a named pipe

I know the difference between > and >> when redirecting to a plain file. But is there a difference when redirecting to a named pipe (fifo)? Assume the namedpipe is a named pipe. Are the ...
2
votes
2answers
62 views

Read a password securely from within process substitution?

Related: Securely feeding a program with a password How to read from stdin in process substitution? Process substitution and cat: can't read stdin I've read the above questions and I still ...
3
votes
2answers
62 views

How to write a “compose” function of bash pipelines with named pipes

This page details psuedocode for creating a compose function that takes n commands and executes them in a pipeline: We program a command compose such that compose cmd1 cmd2 ... cmdn behaves ...
1
vote
0answers
25 views

On-the-fly decompression

I have 44 TB worth of gzipped files that are 1.5 GB each when compressed or 2.0 GB when uncompressed. I have a tool that can read only uncompressed files. I would like to avoid the overhead of ...
1
vote
1answer
49 views

Broken pipe error when reading stops

I'm trying to make a quick & dirty bash function to compare the audio content of two FLAC files without decoding them into a temp file. With the flac command line tool I'm decoding each file into ...
10
votes
3answers
648 views

How to implement “generators” like $RANDOM?

The special variable $RANDOM has a new value every time it's accessed. In this respect, it is reminiscent of the "generator" objects found in some languages. Is there a way to implement something ...
-1
votes
1answer
84 views

are named pipes (mkfifo) the predecessor of RabbitMQ? [closed]

Is RabbitMQ, for inter process communication, like pipes and named pipes? How does RabbitMQ compare to named pipes? Except distributed systems. (RabbitMQ, for those who haven't encountered it, is an ...
5
votes
2answers
117 views

Why process does not (seem to) exist before fifo read

I have a trivial Python script: #!/usr/bin/python import os, sys sys.stderr.write('I am %s' % os.getpid()) sys.stderr.flush() print "hello" sys.stderr.write('I am done') When I run this script ...
1
vote
1answer
74 views

Rsync over custom shell with fifo

I'm trying to use rsync over a custom shell created with fifo. However I cannot find a way to deal with it : The script initsh to simulate a standard shell by fifo : #!/bin/bash rm fin mkfifo fin ...
2
votes
0answers
47 views

Write to a named pipe interactively

I want to use a named pipe as input for a command line utility I'm writing. I know I can echo "all work and no play" > ./namedpipe to write once to the named pipe, but is there a way to do that ...
11
votes
4answers
385 views

Program output redirection

When trying to redirect program output with the "some number greater than" syntax (eg foo 2> myfile), what are the possible numbers here and what do they represent? I believe 1 is /dev/stdout, 2 ...
1
vote
1answer
144 views

How to force ssh-add to ask passphrase when loading RSA key from FIFO?

I've created the following RSA key (with passphrase): ssh-keygen -t rsa -f rsa_foo Now when adding the key in normal way, ssh-add asks for the passphrase fine: $ ssh-add rsa_foo Enter passphrase ...
2
votes
0answers
188 views

named pipe proxy over SSH

I am used to forwarding a remote service port on localhost using ssh like: ssh -L 2181:localhost:2182 user@server (forward remote host port 2182 to local port 2181) now, from the machine I ssh to, I ...
8
votes
1answer
986 views

Increase FIFO size limit

Thanks to the answers to my other question, I now understand that FIFO on Linux, i.e. /dev/xconsole has a buffer limit of 64 KB. How can I increase this limit to 128 KB? Apparently, I will need to ...
0
votes
0answers
67 views

Is the pipe overloaded?

I am using a coprocess to run commands in siebel server manager. I am reading commands from a text file (around 2000 commands) and executing them one by one in siebel server manager. After executing ...
1
vote
2answers
344 views

run command with sudo su in script and redirect stdin and stdout

I need to run command as another user and write to its stdin and read from its stdout. I can run command as another user only via opening new shell: sudo su anotheruser but with new opened shell i ...
-2
votes
2answers
575 views

Pipe Named Fifo [closed]

I am implementing a named pipe to write and read data between two processes in cpp. The first process acquires a feature point from an image in real time while the second reads this feature point. It ...
4
votes
1answer
81 views

Joined pipelines

Considering a routine such as this one: alpha() { echo a b c |tr ' ' '\n'; } which outputs a stream, I would like to take the output stream, transform it, and paste it with the original output ...
5
votes
3answers
415 views

How to forward between processes with named pipes?

The /tmp/in, /tmp/out and /tmp/err are named pipes, already created and opened by some process (for reading, writing and writing, respectively). I would like to create a new process that pipes its ...
6
votes
3answers
516 views

Why does `cat` seem to iterate over JSON arrays?

I'm puzzled by this behaviour of cat when trying to output a heredoc containing JSON in bash 3.2: input: $ cat <(cat <<EOF > {"x":[{"a":1,"b":2}]} > EOF) output: {"x":["a":1]} ...
3
votes
3answers
234 views

Output to stdout works, fails when redirected to a file or a fifo

I have a Wifi antenna and I want to optimise it's position. For that propose I'd like to visualise the current latency in real time. I wrote a one-line script to output the ping latency: lat.sh: ...
3
votes
2answers
942 views

How do I close a FIFO pipe?

This script: #!/bin/bash tmppipe=/tmp/temppipe mkfifo $tmppipe echo "test" > $tmppipe cat $tmppipe exit Does not terminate. I assume that the cat command is waiting for an EOF from the pipe; how ...
3
votes
2answers
132 views

Including process output in tar file

I have a simple bash script to generate a compressed tar file and upload it to an external server for backup purposes. In addition to files on the server, I'd like to include dumps of a few mysql ...
3
votes
1answer
216 views

BASH: Fifo troubles - seems selective about input

So I am having a problem with a BASH service in Debian 7 that I've been working on for quite a while and that randomly started having trouble with its fifo, or so it seems. It is based on kind of your ...
3
votes
1answer
288 views

Using named pipes to send keypresses to an interactive program

I have a program, mpg123, which has an interactive mode that allows keyboard commands from stdin to do useful things such as control volume. I am trying to start up mpg123 so that it reads commands ...
1
vote
1answer
76 views

How to create dynamic process substitution based on the input (multiple FIFOs)?

I'm trying to create multiple process substitutions based on the output from the other command, however I'm not sure how I can achieve that. As sample scenario, I've given list of files: touch a.txt ...
2
votes
1answer
235 views

Printing from a FIFO pipe without interfering?

I have a couple of C programs sending messages back and forth through FIFO named pipes in Unix, and I want to see what's going through the pipes using an external program. The problem is that if I use ...
1
vote
0answers
130 views

Correct/Efficient/Reliable way to import log file entries to a MySQL database?

I'd like to be able to query our server access log files to identify abuse patterns. Log files really aren't very queryable, and it would be fairly easy to do what I wanted if each hit was a row in a ...
14
votes
9answers
1k views

Filter or pipe certain sections of a file

I have an input file with some sections the are demarcated with start and end tags, for example: line A line B @@inline-code-start line X line Y line Z @@inline-code-end line C line D I want to ...
2
votes
2answers
47 views

Triggering zero-byte reads on FIFO/pty

I'd like to know how to emulate the ICANON behavior of ^D: namely, trigger an immediate, even zero-byte, read in the program on the other end of a FIFO or PTY or socket or somesuch. In particular, I ...
1
vote
1answer
54 views

Is it possible to link file to an output of command line? [duplicate]

I want to create symlink, or a pipe, or something else that could be used in lieu of a file. I want that pseudo-file to execute a command and output of that command should become the contents of that ...
1
vote
0answers
194 views

'nc' using ~100% CPU when transferring a FIFO file

On the server which is always connected I’m using this in a crontab: @reboot nc -k -l 1337 < /tmp/fifo.fifo On the client which is not always on the network or switched on I’m using this ...
3
votes
2answers
812 views

Named pipes: several experiments leads to confusion [closed]

I've come across various articles and SO questions and I am still confused about something that I use on daily basis, but never realized how confusing it can be. I am experimenting with (named) pipes ...
-1
votes
1answer
77 views

What is the persistence of the handle of a broken FIFO?

Say an application opens for writing, a file, which is a FIFO to which something is “listening”, and later, this “listener” closes; if the application attempts to write to it, the application will get ...
1
vote
1answer
97 views

Non-block poll from two pipes?

Here is what I want to do.. mkfifo infrequentlog frequentlog sort -m infrequentlog frequentlog > mylog funcA > infrequentlog funcB > frequentlog So that when monitor tail, I can see the ...
2
votes
1answer
259 views

Bash Named Pipe - Atomic Reading

I have set up one script to write to a named pipe, then four scripts to read the named pipe. For most of the entries, there are no problems, but when there is little processing to be done on the ...
2
votes
1answer
421 views

Create a named pipe / fifo that executes a command when read

I'm looking to supply a process that only reads from a local file with content from an http URL. The process is a daemon that is hard-coded to read a specific file in it's current working directory ...
17
votes
1answer
1k views

Why is a named pipe as slow as writing to a file?

I'm trying to understand how named pipes work so that I can streamline my one-way interprocess communication. I expect some overhead due to copying data into a circular buffer, which I would have ...
4
votes
1answer
89 views

Why does my named pipe keep getting modified?

I have a named fifo created from a C program like this: res = mkfifo("/home/myfolder/myfifo", 0666); after that there are only reads and writes. Now, from this answer : Does a named pipe change ...
1
vote
0answers
185 views

Can rsyslogd write to more than one named pipe

If I have the following two lines in the rsyslogd.conf *.* |/var/run/rsyslog/pipe1 *.* |/var/run/rsyslog/pipe2 Will all messages be sent to both pipes? I have two programs that want to inspect ...
2
votes
2answers
490 views

How to read from a named pipe after the writer process exits?

I want to use named pipes as asynchrnonous task queues (JMS-like, perhaps). For example, suppose a client sends some data that I wish to insert into a database, but I want the client to disconnect ...
2
votes
1answer
437 views

create several /dev/xconsole files automatically upon boot

I understand from my previous question that the /dev/xconsole file can be created using mkfifo. On my system (Debian Wheezy), /dev/xconsole gets created automatically. I am wondering who creates it ...
2
votes
3answers
442 views

Buffering (named) pipe in GNU OS

In the GNU OS a process can only write data to a pipe if another process reads the same data (from the same pipe) at the same time. Is there something like a pipe which lets the 1st process write and ...
5
votes
1answer
366 views

Why does “cat /tmp/out1 > /tmp/in2 &” appear as “bash” with the “ps” command?

Consider: mkfifo /tmp/out1 mkfifo /tmp/in2 cat /tmp/out1 > /tmp/in2 & When I run ps it appears as bash in the CMD section. Why?
1
vote
1answer
83 views

Custom python REPL with fifos

Is it possible to make a custom REPL (for e.g. python) with two fifos? Explanation of the problem Suppose I have two fifos in and out. I have three terminals. In the first I want to run commands ...