1

I want to be able to write a script that can take stdout as an argument, if anything is piped into it (ultimately, I would like it to be polymorphic) -

The trouble is, I have searched and searched for how to do this with no avail - lots of alternative suggestions about how to do other things that are not - this:

cat /var/log/some.log | grep something | awk '{print $1 $6 $8}' | myscript

Why do that, instead of? : myscript $(!!) At this point, solely to prove that it is possible...

I know that you can 'read variable' in a script, but say I don't care about the lines - let's say, I want to accept the whole of it as a blob of text and do something with it in the script -

Do I really have to :

while read x; do
stdin=$stdin" "$x;
done;

solely in order to read from STDIN ?

There must be a better way ...

2

If you want to read all of stdin into a shell script, usually you just capture it into a temp file:

TMPFILE=$(mktemp /tmp/$0.$$.XXXXXX)
cat > "$TMPFILE"
# Script works with $TMPFILE and its contents,
# ultimately writing everything to stdout.
rm -f "$TMPFILE"

Even system utilities do things very much like this. sort has to have all of stdin before it can print anything to stdout, for example.

  • This seems to be the most simple solution for doing this - and I'm sure this is what I'm going to go with - – rm-vanda Jul 2 '15 at 15:31
3

You can read from stdout by redirecting input from file descriptor 1. Stdout is file descriptor 1 by definition. The fact that file descriptor 1 is used for output is a matter of convention, not a technical obligation. However it's a bizarre thing to do which is bound to confuse the people who use your script.

read line <&1

If you want to read a whole file, use cat. To stuff the content into a variable, use a command substitution.

whole_input=$(cat)
whole_input_from_stdout=$(cat <&1)

Some shells let you write $(</dev/stdin) as a slightly more efficient shortcut for $(cat).

Note that this strips trailing newlines. This behavior is built into command substitution. To retain trailing newlines, hide them behind another character and remove that character.

whole_input=$(cat; echo .); whole_input=${whole_input%?}

The shell variable will only contain the data up to the first null byte, if there is a null byte in the input. With some shells, you'll get all the data with the null bytes stripped. Shells can't deal with binary data. Zsh is an exception, it retains the null bytes.

  • while this is certainly the most elegant solution, it seems that that method does not actually preserve the newlines - - - – rm-vanda Jul 2 '15 at 15:31
  • @rm-vanda $(cat) preserves the newlines except for the trailing ones. The two-step process I describe afterwards preserves all newlines. – Gilles Jul 2 '15 at 15:43
2

You don't have to get the whole lump if you don't want to - you can chunk out stdin and work it as a stream if you like. I did some googling, though, and I don't think I can offer you any advice on how to change the sex of your script, or whatever it is you meant by polymorphic.

In any case, I typically find that putting the whole of input aside in some storage is rarely what I'm after. I usually want to handle it in delimited lumps of some kind. Here's an example of how you can get input of any kind split out into 4k chunks per while loop iteration:

splitin(){
    dd obs=4k | { j=$1 f=$2;shift 2 && 
    while dd bs=4k count=1 of="$f"  &&
          [ -s "$f" ] 
    do    "$j" "$@" < "$f"
    done
};  }

...which is a function you might call from your shell script with the name of a some job you want to perform on 4k input intervals and the name of a temp file it can use to store each latest chunk. Like:

splitin handle_chunk /tmp/work/chunk \
        other args to pass on as appropriate   
  • Awesome. I will spend time dissecting this - but this is a bit more complex than what I was looking for - But thanks! this is a really good answer... – rm-vanda Jul 2 '15 at 15:43
  • 1
    @rm-vanda what are you looking for? and why do you want to give your script a sex-change? – mikeserv Jul 2 '15 at 15:44
  • Hah! I wasn't really looking to read it in chunks - the answer i accepted was precisely what I was looking for - But this snippet is quite complex, and I will break it down and learn from it So thank you ! – rm-vanda Jul 6 '15 at 13:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.