29

This question already has an answer here:

Suppose a program cook takes one argument: the pathname of a text file containing the recipe of the food to cook. Suppose I wish to call this program from within a bash script, also suppose I already have the recipe in a string variable:

#!/bin/bash

the_recipe="$(cat << EOF
wash cucumbers
wash knife
slice cucumbers
EOF
)"

cook ...  # What should I do here? It expects a file, but I only have a string.

How can I pass the recipe to the command when it expects a filename argument?

I thought about creating a temporary file just for the purpose passing a file, but I wish to know if there are alternative ways to solve this problem.

marked as duplicate by Kusalananda bash Mar 13 at 14:10

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • The duplicate question talks about data arriving on a pipe. This question talks about having data in a variable. The code in this question may trivially be reformulated by adding printf '%s\n' "$the_recipe" | cook, and this would make it an exact duplicate. I chose to mark this as a dupe of the other as it was older and as the other was more generally formulated from the start. – Kusalananda Mar 13 at 14:15
40

You can use the "fake" filename /dev/stdin which represents the standard input.

So execute this:

echo "$the_recipe" | cook /dev/stdin

The echo command and the pipe sends the contents of the specified variable to the standard input of the next command cook, and that opens the standard input (as a separate file descriptor) and reads that.

  • 2
    Is this method limited to commands that take only one argument? If cook takes two file arguments, and I have the_recipe1 and the_recipe2 (both of which are string variables), will this method still work? – Flux Mar 12 at 10:18
  • 5
    @Flux You can do it with /dev/fd but that's beyond the scope of your initial question. But I think you'll find examples of that in unix.stackexchange.com/questions/tagged/file-descriptors – Gilles Mar 12 at 10:24
  • This is not 100% portable, though it does come pretty close in practice: unix.stackexchange.com/q/36403/91565 – Kevin Mar 12 at 21:38
37

One other way to use the process-substitution feature of the bash shell causes a FIFO to be created under /tmp or /var/tmp, or uses a named file descriptor (/dev/fd/*), depending on the operating system. The substitution syntax (<(cmd)) is replaced by the name of the FIFO or FD, and the command inside it is run in the background.

cook <(printf '%s\n' "${the_recipe}")

The command cook now reads the content of the variable as if it were available in a file.

This could also be used for more than one input file:

cook <(printf '%s\n' "${the_1st_recipe}") <(printf '%s\n' "${the_2nd_recipe}")
  • 4
    Alternatively, cat <<<"$the_recipe" inside the process substitution. – Kusalananda Mar 12 at 10:25
  • 5
    Note that process substitution is a ksh feature (now also supported by zsh and bash). Chances are cook will expect a text file, so you should probably make it '%s\n' instead of '%s'. See also <(<<<$the_recipe) in zsh. – Stéphane Chazelas Mar 12 at 10:30
19

It depends on the application's capabilities. It may or may not insist on a named file, and it may or may not insist on a seekable file.

Anonymous pipe

Some applications just read input from anywhere. Typically they default to reading from standard input. If you have the data in a string and you want to pass it to the application's standard input, there are a few ways. You can pass the information via a pipe:

printf '%s' "$the_recipe" | cook     # passes $the_recipe exactly
printf '%s\n' "$the_recipe" | cook   # passes $the_recipe plus a final newline

Don't use echo here because depending on the shell, it might mangle backslashes.

If the application insists on a file argument, it might accept - to mean standard input. This is a common convention, but not systematic.

printf '%s\n' "$the_recipe" | cook -

If the application needs an actual file name, pass /dev/stdin to mean standard input.

printf '%s\n' "$the_recipe" | cook /dev/stdin

For some applications, even this is not enough: they need a file name with certain properties, for example with a given extension, or they need a file name in a writable directory where they create temporary files. In such cases, you generally need a temporary file, although sometimes a named pipe is enough.

Named pipe

It's possible to give a pipe a name. I mention this for completeness, but it's rarely the most convenient way. It does avoid getting the data on disk. It's suitable if the application needs a file with constraints on the name, but doesn't need the file to be seekable.

mkfifo named.pipe
printf '%s\n' "$the_recipe" >named.pipe & writer=$!
cook named.pipe
rm named.pipe
wait "$writer"

(Error checking omitted.)

Temporary file

If the application needs a seekable file, you need to create a temporary file. A seekable file is one where the application can go back and forth, reading arbitrary portions at a time. A pipe doesn't allow this: it needs to be read from start to finish, in sequence, without ever going backwards.

Bash, ksh and zsh have a convenient syntax to pass a string as input via a temporary file. Note that they always append a newline to the string (even if there's already one).

cook <<<"$the_recipe"

With other shells, or if you want to control which directory contains the temporary file, you need to create the temporary file manually. Most unices provide a mktemp utility to create the temporary file securely. (Never use something like … >/tmp/temp.$$! It allows other programs, even running as other users, to hijack the file.) Here's a script that creates a temporary file and takes care of removing it if interrupted.

tmp=
trap 'rm -f "$tmp"' EXIT
trap 'rm -f "$tmp"; trap "" HUP; kill -INT $$' HUP
trap 'rm -f "$tmp"; trap "" INT; kill -INT $$' INT
trap 'rm -f "$tmp"; trap "" TERM; kill -INT $$' TERM
tmp=$(mktemp)
printf '%s\n' "$the_recipe" >"$tmp"
cook "$tmp"

To choose where to create the temporary file, set the TMPDIR environment variable for the mktemp call. For example, to create the temporary file in the current directory rather than the default location for temporary files:

tmp=$(TMPDIR="$PWD" mktemp)

(Using $PWD rather than . gives you an absolute file name, which means you can safely call cd in your script without worrying that $tmp will no longer designate the same file.)

If the application needs a file name with a certain extension, you need to create a temporary directory and create a file there. To create the temporary directory, use mktemp -d. Again, set the TMPDIR environment variable if you want to control where the temporary directory is created.

tmp=
trap 'rm -rf "$tmp"' EXIT
trap 'rm -rf "$tmp"; trap "" HUP; kill -INT $$' HUP
trap 'rm -rf "$tmp"; trap "" INT; kill -INT $$' INT
trap 'rm -rf "$tmp"; trap "" TERM; kill -INT $$' TERM
tmp=$(mktemp -d)
printf '%s\n' "$the_recipe" >"$tmp/myfile.ext"
cook "$tmp/myfile.ext"
  • I'm pretty sure you can use a here-doc as input, too (I think you might have to use <(cat <<END) or similar; it's years since I've done that). – Toby Speight Mar 12 at 13:17
1

Always check if your version of cook supports the argument - (dash) as a filename before trying more complicated solutions. If this convention is supported, it tells cook to read from the standard input, like so:

echo "$the_recipe" | cook -

or

cook - <<<"$the_recipe"

Not the answer you're looking for? Browse other questions tagged or ask your own question.