Tweeted twitter.com/StackUnix/status/1070966287235452928
2 added 1 character in body
source | link

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not be necessary.

An example of what I would want my script to do:

$ echo '
> Line 1
> Line B
> Line iii' | ./myscript.sh 'sed s/B/b/g' 'sed s/iii/III/' 'cat'
1:Line B     2:Line b
1:Line iii   3:Line III

So far I have this:

i=0
SOURCES=()
TARGETS=()

for c in "$@"; do
    SOURCES+=(">($c > tmp-$i)")
    TARGETS+=("tmp-$i")
    i=$((i+1))
done

eval tee ${SOURCES[@]} >/dev/null <&0
comm ${TARGETS[@]}

The issues are:

  • There seems to be a race condition. By the end of execution comm tmp-0 tmp-1 has the desired output (more-or-less) but when executed from the script the output seems non deterministic-deterministic.
  • This is limited to just 2 inputs, but I need at least 3 (ideally any number)
  • This creates temporary files that I would have to keep track of and delete afterwards, an ideal solution would only use redirection

The constraints are:

  • The input may not be ending. In particular the input could be something like /dev/zero or /dev/urandom, so merely copying the input to a file wontwon't work.
  • The commands may have spaces in them and be fairly complicated themselves
  • I want a line-by-line, in-order comparison.

Any idea how I could go about implementing this.? I basically want something like echo $input | tee >(A >?) >(B >?) >(C >?) ?(compare-all-files) if only such a syntax existed.

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not be necessary.

An example of what I would want my script to do:

$ echo '
> Line 1
> Line B
> Line iii' | ./myscript.sh 'sed s/B/b/g' 'sed s/iii/III/' 'cat'
1:Line B     2:Line b
1:Line iii   3:Line III

So far I have this:

i=0
SOURCES=()
TARGETS=()

for c in "$@"; do
    SOURCES+=(">($c > tmp-$i)")
    TARGETS+=("tmp-$i")
    i=$((i+1))
done

eval tee ${SOURCES[@]} >/dev/null <&0
comm ${TARGETS[@]}

The issues are:

  • There seems to be a race condition. By the end of execution comm tmp-0 tmp-1 has the desired output (more-or-less) but when executed from the script the output seems non deterministic.
  • This is limited to just 2 inputs, but I need at least 3 (ideally any number)
  • This creates temporary files that I would have to keep track of and delete afterwards, an ideal solution would only use redirection

The constraints are:

  • The input may not be ending. In particular the input could be something like /dev/zero or /dev/urandom, so merely copying the input to a file wont work.
  • The commands may have spaces in them and be fairly complicated themselves
  • I want a line-by-line, in-order comparison.

Any idea how I could go about implementing this. I basically want something like echo $input | tee >(A >?) >(B >?) >(C >?) ?(compare-all-files) if only such a syntax existed.

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not be necessary.

An example of what I would want my script to do:

$ echo '
> Line 1
> Line B
> Line iii' | ./myscript.sh 'sed s/B/b/g' 'sed s/iii/III/' 'cat'
1:Line B     2:Line b
1:Line iii   3:Line III

So far I have this:

i=0
SOURCES=()
TARGETS=()

for c in "$@"; do
    SOURCES+=(">($c > tmp-$i)")
    TARGETS+=("tmp-$i")
    i=$((i+1))
done

eval tee ${SOURCES[@]} >/dev/null <&0
comm ${TARGETS[@]}

The issues are:

  • There seems to be a race condition. By the end of execution comm tmp-0 tmp-1 has the desired output (more-or-less) but when executed from the script the output seems non-deterministic.
  • This is limited to just 2 inputs, but I need at least 3 (ideally any number)
  • This creates temporary files that I would have to keep track of and delete afterwards, an ideal solution would only use redirection

The constraints are:

  • The input may not be ending. In particular the input could be something like /dev/zero or /dev/urandom, so merely copying the input to a file won't work.
  • The commands may have spaces in them and be fairly complicated themselves
  • I want a line-by-line, in-order comparison.

Any idea how I could go about implementing this? I basically want something like echo $input | tee >(A >?) >(B >?) >(C >?) ?(compare-all-files) if only such a syntax existed.

1
source | link

Pass input to multiple commands and compare their outputs

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not be necessary.

An example of what I would want my script to do:

$ echo '
> Line 1
> Line B
> Line iii' | ./myscript.sh 'sed s/B/b/g' 'sed s/iii/III/' 'cat'
1:Line B     2:Line b
1:Line iii   3:Line III

So far I have this:

i=0
SOURCES=()
TARGETS=()

for c in "$@"; do
    SOURCES+=(">($c > tmp-$i)")
    TARGETS+=("tmp-$i")
    i=$((i+1))
done

eval tee ${SOURCES[@]} >/dev/null <&0
comm ${TARGETS[@]}

The issues are:

  • There seems to be a race condition. By the end of execution comm tmp-0 tmp-1 has the desired output (more-or-less) but when executed from the script the output seems non deterministic.
  • This is limited to just 2 inputs, but I need at least 3 (ideally any number)
  • This creates temporary files that I would have to keep track of and delete afterwards, an ideal solution would only use redirection

The constraints are:

  • The input may not be ending. In particular the input could be something like /dev/zero or /dev/urandom, so merely copying the input to a file wont work.
  • The commands may have spaces in them and be fairly complicated themselves
  • I want a line-by-line, in-order comparison.

Any idea how I could go about implementing this. I basically want something like echo $input | tee >(A >?) >(B >?) >(C >?) ?(compare-all-files) if only such a syntax existed.