I'm sure it ought to be possible. I can only suggest a hack where
you use extra ssh connections to each carry another pair of file descriptors.
E.g. the following proof of concept script does a first ssh to run a dummy command (sleep) to connect
up local fds 5 and 6 to remote stdin and stdout, presuming these fds are
the ones you want to add to the usual 0,1,2.
Then the real ssh is done, and on the remote it connects up remote fds 5 and
6 to the stdin and stdout of the other ssh.
Just as an example, this script passes a gzipped man page
through to the remote, which unzips and runs it through man.
The stdin and stdout of the real ssh are still available for
exec 5</usr/share/man/man1/ssh.1.gz 6>/tmp/out6 # pretend need 5 and 6
ssh remote 'echo $$ >/tmp/pid; exec sleep 99999' <&5 >&6 &
sleep 1 # hack. need /tmp/pid to be set
ssh remote '
exec 5</proc/$pid/fd/0 6>/proc/$pid/fd/1
gzip -d <&5 | man /dev/stdin >&6
kill -hup $pid