The following example is kind of classical way of using pipes. On right side of pipe we have a "simple" utility which reads data from stdin (pipe) and prints to stdout in reverse order:
{ cat <</EOF
Hello
World
/EOF
} | rev
Obviously the result is:
olleH
dlroW
Now I want to feed that data through pipe into a command/utility, which is part of a shell here-doc:
{ cat <</EOF
Hello
World
/EOF
} | ksh -x <</EOF
rev
/EOF
This does not work, no error message, nothing, piped data gets lost since ksh is redirecting stdin to here-doc.
I tried some kind of redirection:
set -x
{ cat <</EOF
Hello
World
/EOF
} | { exec 4<&0; ksh -x <</EOF; exec 0<&4 4<&-; }
rev <&4
/EOF
But this does not work either, it shows an error on rev command, obviously the redirection of stdin to fd4 is not passed through to subprocesses of ksh:
+ exec
+ 4<& 0
+ ksh -x
+ cat
+ 0<< \/EOF
rev <&4
/EOF
+ 0<< \/EOF
Hello
World
/EOF
+ rev
+ ksh[1]: 4: cannot open [Ungültiger Dateideskriptor]
+ exec
+ 0<& 4 4<& -
I'm stuck here.
My real application is much more complex: on left side of pipe I will do ssh to a source server, optionally sudo to a different user on that server and run tar to pipe some data files. On the right side of pipe I will have to ssh to a target server, optionally sudo to different user and finally run tar to receive the data from pipe.
Btw.: everything works fine if I do NOT make use of the here-doc on right side of pipe:
{ cat <</EOF
Hello
World
/EOF
} | ksh -xc "rev"
Even with ssh plus sudo plus tar this works fine. For some reason I would prefer to have a here-doc for the shell on the right side. As you can see ksh is my favourite shell within that application.
As requested: my original statement is like
ssh $SSHOPT $INST_FILE_USER@$INST_FILE_HOST "cd $IDIR && { tar -cvf - $IFIL && echo RCSRC0 >&2 || echo RCSRC1 >&2; }" 2>$TMPFILE.err.src | ssh $SSHOPT $TRG_USER@$TRG_HOST "cd $TDIR && { tar -xmvf - && echo RCTRG0 || echo RCTRG1; }" 1>$TMPFILE.out.trg 2>&1
It is working fine so far. As you can see, there is a minimum of error checking implemented, which allows for recognition of tar success. Now I must add sudo functionality, that means between the shell from ssh and cd command I must add some code to switch user. Is it really sudo, which will be used? Might be pbrun command since customers might prefer pbrun. These different scenarios can be dealt with much more flexible if here-docs are in place instead of 1 monster statement.
Update from May 21st: I tried to follow recommendations given so far, but I'm still stuck. Let's take a look at the answer given below, which is working fine for me:
echo yup | ksh -c 'exec 4<&0; ksh -c "rev <&5" 5<&4'
puy
If I start using ssh or using ssh combined with sudo my sample with redirection does not work any more:
echo yup | ssh admin@trg14 ksh -c 'exec 4<&0; ksh -c "rev <&5" 5<&4'
bash: 4: Bad file descriptor
echo yup | ssh admin@trg14 sudo -n -u trg4 ksh -c 'exec 4<&0; ksh -c "rev <&5" 5<&4'
bash: 4: Bad file descriptor
I have tried 2 scenarios with ssh and sudo combination: first one is using the here-doc as input to ksh:
Scenario 1:
echo yup | ssh admin@trg14 sudo -n -u trg4 -- ksh<</EOF
whoami && whoami && rev && whoami
/EOF
trg4
trg4
trg4
This is working so far. Note: all 4 commands are run as user trg4, rev does not get any input through pipe as expected due to the here-doc.
Therefore I tried to implement the solution suggested below: here-doc will be input as parameter to ksh:
Scenario 2:
echo yup | ssh admin@trg14 sudo -n -u trg4 -- ksh -c "$(cat<</EOF
whoami && whoami && rev && whoami
/EOF
)"
trg4
admin
puy
admin
The output from rev is as expected, but: whoami is showing us that in this version/implementation only the first command is run as sudo user, the remaining commands will be run as ssh login user! In case of rev no problem, but if you need to access or write to database files you will fail with permission issues.
Conclusion from these 2 samples: scenario 1 might be the right track to run all commands under the correct sudo user, but it will need a solution to the redirection of stdin in order to be able to use the pipe.
Any help appreciated!
cat > /tmp/x
and then type some more text, how does the shell know that this extra text is not more commands rather than input to the cat command? In yourredirection
example which fails, does it still fail if you change the RHS to use()
instead of{}
and move therev <&4
inside the parenthesis?/dev/fd/...
in your operating system or 2) process substitution<(...)
in your outer shell?