I have few functions defined in this fashion:

function f {
  read and process $1
  echo $result

I want to compose them together so that invocation would look like f | g | h.

What idiom shoud I use to convert function working on arguments to one reading arguments from stdin? Is it possible to read pairs, tuples of arguments from stream without need to escape them (e.g. null terminating)?

  • Either you want something like h(g(f(...))) or each of the functions reads from standard input (read x; ...) and writes to standard output (echo ...). – vonbrand Feb 8 '14 at 0:22

One potential approach would be to put a while...read construct inside your functions which would process any data that came into the function through STDIN, operate on it, and then emit the resulting data back out via STDOUT.

function X {
  while read data; do

Care will need to be spent with how you configure your while ..read.. components since they'll be highly dependent on the types of data they'll be able to reliably consume. There may be an optimal configuration that you can come up with.


$ logF() { while read data; do echo "[F:$(date +"%D %T")] $data"; done; }
$ logG() { while read data; do echo "G:$data";                    done; }
$ logH() { while read data; do echo "H:$data";                    done; }

Here's each function by itself.

$ echo "hi" | logF
[F:02/07/14 20:01:11] hi

$ echo "hi" | logG

$ echo "hi" | logH

Here they are when we use them together.

$ echo "hi" | logF | logG | logH
H:G:[F:02/07/14 19:58:18] hi

$ echo -e "hi\nbye" | logF | logG | logH
H:G:[F:02/07/14 19:58:22] hi
H:G:[F:02/07/14 19:58:22] bye

They can take various styles of input.

#-- ex. #1
$ cat <<<"some string of nonsense" | logF | logG | logH
H:G:[F:02/07/14 20:03:47] some string of nonsense

#-- ex. #2    
$ (logF | logG | logH) <<<"Here comes another string."
H:G:[F:02/07/14 20:04:46] Here comes another string.

#-- ex. #3
$ (logF | logG | logH)
Look I can even
H:G:[F:02/07/14 20:05:19] Look I can even
type to it
H:G:[F:02/07/14 20:05:23] type to it
H:G:[F:02/07/14 20:05:25] live
H:G:[F:02/07/14 20:05:29] via STDIN
..type Ctrl + D to stop..

#-- ex. #4
$ seq 5 | logF | logG | logH
H:G:[F:02/07/14 20:07:40] 1
H:G:[F:02/07/14 20:07:40] 2
H:G:[F:02/07/14 20:07:40] 3
H:G:[F:02/07/14 20:07:40] 4
H:G:[F:02/07/14 20:07:40] 5

#-- ex. #5
$ (logF | logG | logH) < <(seq 2)
H:G:[F:02/07/14 20:15:17] 1
H:G:[F:02/07/14 20:15:17] 2

As an addendum to slm's answer, I did some experimenting with null-separated tuples as function arguments:

$ sayTuple() { 
    IFS= read -r -d $'\0' d1
    IFS= read -r -d $'\0' d2
    echo "sayTuple: -$d1- -$d2-"

Notes: sayTuple twice reads a null-terminated record -d $'\0' handling any space surrounding input IFS=. echo back records surrounded by -

Result shows it correctly handles null-terminated input containing \n and \t:

$ printf "%s\0%s\0" "Hello " $' Brave\n\tWorld' | sayTuple 
sayTuple: -Hello - - Brave

Please add suggestions for improvement in comments, it's an interesting topic.

  • +1 like your idea. We could put a loop inside instead which would allow it to take arbitrary #'s of args. sayTuple() { arr=() ; while IFS= read -r -d $'\0' arg; do arr+="$arg"; done; echo "sayTuple: ${arr[@]}"; }. – slm Feb 9 '14 at 15:51
  • Seems like you should be able to do a IFS= read -r -d $'\0' -a arg but I couldn't get this to work. This would allow for the removal of the while, which seems unnecessary, but was the only pattern I could get to work. – slm Feb 9 '14 at 15:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.