4

This question already has an answer here:

I have a script that looks like the following:

ssh host command1
scp file1 host:/tmp
ssh host command2

Is there a way to combine them all in one ssh/scp call? The problem I have is combining copying files with executing commands, which has to be done interleaved. Was hoping for something like:

ssh_sometool host -c command1 -cp file1 /tmp -c command2

It's not about the syntax ( I could write a function like that), it's about everything happening within the same ssh session

marked as duplicate by Gilles ssh Nov 23 '17 at 17:33

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • what is the problem with first option ? you have to type password 3 times ? – Archemar Nov 23 '17 at 12:26
  • @Archemar, no, the keys are properly configured so no pw is needed, it just fells cumbersome to open and close a new ssh session for each line. Imagine there would be 100 ssh/scp in the script – Chirlo Nov 23 '17 at 12:28
  • @Chirlo If you have to transfer a lot back and forth I'd recommend you to use provisioning tool of some sort. It would make things less cumbersome as most of such tools optimize a number of connections made. Please take a look at my answer. – ddnomad Nov 23 '17 at 13:41
4

First of all, there are no straightforward or standard way to do what you want. Additionally, let me clarify: there is no way to do what you want with a default ssh client.

There is an extremely old fork ssh-xfer. Note that even the developers of this tool emphasize that their solution is "hackish" and really old so you should not use it. The last SSH version that was patched for being usable with this tool is OpenSSH-3.8.1p1 (on Arch Linux the current version is OpenSSH_7.6p1).

If it's important for you to keep things quick minimizing the number of connections/handshakes done between hosts you might look into some kind of provisioning/orchestration tools. Ansible seems to be the easiest way to accomplish the task without much of an additional configuration while Salt might be the quickest.

Anyway both Ansible and Salt utilize additional connections to make things work and I'm not sure you'd win something performance-wise using them instead of ssh/scp combination.

UPDATE: If you have to push and pull a lot of files with some additional scripts running between these transfers almost any provisioning tool (Chef, Puppet, Salt, Ansible) will do the thing quicker than plain ssh/scp.

3

SSH allows to have a session alive for a while after you've disconnected, so you don't have to initiate the connection next time. Here are the options for ssh config to enable it:

Host *
ControlMaster auto
ControlPath ~/.ssh/sockets/%r@%h-%p
ControlPersist 30

ControlMaster enables the sharing of multiple sessions over a single network connection. When set to auto, each new attempt checks if the respective connection exists and either uses it or creates new connection.

ControlPath specifies the path where connection are stored. Make sure that this path exists! That is, create the directory (sockets here, but name it as you want) manually.

ControlPersist specifies the how long the connection lives after you have disconnected. It accepts time in seconds.

For further clarification on the options use man ssh_config.

After you add these to your ~/.ssh/config, only one session will be opened for these three commands.

On my computer it gives 5x speedup: 100 connections in row to some (nearby) host work in 30s without persistent sessions and only 5-6s with them.

P.S. Sometimes (for example, if you changed the network) your existing ssh connections make break and new ones won't establish because the file with the connection does exist. In this case just go to ~/.ssh/sockets and delete the necessary file manually.

1

You can join and pass all your files thorough ssh standard input and receive them in your remote session, I mean something like this:

tar -cf - file1 file2 | ssh host 'TDIR=`mktemp -d`; tar -xf - -C "$TDIR"; cat "$TDIR/file1"; command1; command2 ...'


  • The cat "$TDIR/file1" command is just an example how to access your received files;
  • Use tar with -j or -J switch both on local and remote side to pass your files compressed;
  • You can put all your commands in a bash script, include it in your archive and execute on remote side like ...tar -xf - -C "$TDIR"; bash "$TDIR/script.sh"'
1

Try this

cat file1 | ssh host 'command1 && cat >> /tmp/file1 && command2'
-1

In your script, once the first line runs...

ssh host command1

Execution is handed off to ssh + your second command (scp) will only run when command1 finishes. One way to have these run in parallel, is background each task, like this...

ssh host command1 &
scp file1 host:/tmp &
ssh host command2 &

So now all three tasks run in background, concurrently.

  • maybe OP want command1 to end before copying ? – Archemar Nov 23 '17 at 12:25
  • This answer doesn't have too much to do with the question, edited to clarify – Chirlo Nov 23 '17 at 12:30

Not the answer you're looking for? Browse other questions tagged or ask your own question.