3

I have some arbitrary data as an output of a command (e.g. a fragment of a binary data put out using tail). I want to feed it to another command/program which allows only a file as an argument (e.g. rawtopng [filein] [fileout]), does not support dash (-) argument, and I cannot make a temporary file in between. How can I do it in bash?

Will something like rawtopng <(tail myfile) fileout work?

3

The process substitution rawtopng <(tail myfile) fileout creates a pipe (just like tail myfile | rawtopng - fileout), and passes a name for the pipe to the program. On Linux, rawtopng will see a name like /dev/fd/42, and if it queries the file type, it'll be told that it's a pipe. No write to the filesystem is required (this is an anonymous pipe, not a named pipe). You can get a glimpse of what's going on by running a command like ls -ld <(echo foo)

Will this work? It depends what the rawtopng command requires (which I don't know, I'm not familiar with that command). If all it does is read from the file, that's fine. If it needs to seek in the file (i.e. if it doesn't read its input linearly from start to end), or if it needs to know the size of the input before it starts working, that won't work. In such cases, you will need a temporary file.

If the command needs seekable input or input with a size that's known in advance, then you don't have a choice, you need to create a temporary file. If you want to avoid writing to disk, you can arrange to put the file on a tmpfs filesystem, if your Unix variants offers that (Linux does). You can use the mktemp command to create a temporary file:

tmp=$(TMPDIR=/tmp mktemp XXXXXXXXXXXX.raw)
rawtopng "$tmp" fileout
rm "$tmp"

In zsh, the process substitution variant rawtopng =(tail myfile) fileout creates a temporary file and remove it once the command has finished.

Another case where process substitution won't cut it is if the program insists on a file with a particular extension. In this case, creating a named pipe might help:

mkfifo /tmp/foo.raw
tail myfile >/tmp/foo.raw &
rawtopng /tmp/foo.raw fileout
rm /tmp/foo.raw
  • 1
    Thanks! Note that not every program supports - as an argument. Will tail myfile | rawtopng /dev/stdin fileout work in such case too? – niutech Feb 1 '16 at 10:54
  • @niutech Yes, it works without any special support in the program. – Gilles Feb 1 '16 at 11:49
2

I think you're on the right track! <(tail myfile) will create an Anonymous Named Pipe, and is a type of Bash Process Substitution.

Normally, this will pass /dev/fd/XX as the "filename", which is a file descriptor interface to the running process.

From the examples in the linked documentation:

bash$ wc <(cat /usr/share/dict/linux.words)
 483523  483523 4992010 /dev/fd/63

To read from a process, use <(command), and to write to a process, use >(command).

You can read more about Process Substitution on Wikipedia as well.

0

You can create a named pipe.

Do something like mkfifo input to create the pipe.

Start the program that you want to read the output of, and redirect the output to the pipe, aka myprog > input

Then run your other command with input as the file.

Note that pipes are one directional. The program will be able to read, but not to write.

  • I just want to read from a pipe. But do I need to make a named pipe? Can't I use an anonymous pipe? – niutech Jan 31 '16 at 0:39
  • Well, I think an anonymous named pipe should be enough. – niutech Jan 31 '16 at 0:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.