So I'm trying to transfer a bunch of files via SCP. Some of these are too large to be stored on the recipient (Android phone, 4GB file size limit).

The sender is almost out of space, so I can't create intermediate files locally.

I'd like to tar up the bunch and stream it through split so that I can get smaller segments that'll be accepted by the phone, i.e. local command:

tar -cvf - ~/batch/ | split --bytes=1024m - batch.tar.seg

But I'm not sure how I'd pipe that into scp to get it the phone. According the comment on this post, it's possible, but I first of all don't quite get what he's saying, second of all I'm not sure how to accomplish this as there'll be multiple files output from split.

Any ideas?

3 Answers 3


If the recipient has split, then you can do:

tar -cvf - ~/batch/ | gzip |
  ssh recipient 'cd /destination &&
    split --bytes=1024m - batch.tar.gz.seg'
  • It does! And it works! What a great way to solve it :) Thank you! Feb 8, 2013 at 8:27

While the comment you are mentioning does refer to scp and ssh reading from the STDIN, this is not exactly your case here. The reason is the split: it does not write to STDOUT. So even if you add more pipes, you will not get the data there. Stephane suggests the right thing in his answer above: don't store anything on the source as it has lack of free space, just package, gzip and transfer the data; and then do the splitting on the receiving end, where the chunk size matters.

  • Thank you for that extra info - I hadn't thought of that, of course you're right :) Feb 8, 2013 at 8:28
  • You're welcome, DanielSmedegaardBuus!
    – Leonid
    Feb 8, 2013 at 10:11

Your referenced post describes the problem well: As you are using a pipe you do not copy files but rather redirect standard streams. And scp is not the right tool for that (it is for copying files), ssh is the right program: You redirect the stream to ssh standard input and this will output that input onto the remote host and feed it to the standard input of a specified command:

local$ tar -cvzf - ~/batch/ | ssh target 'split --bytes=1024m - batch.tar.seg'

Note that I added the compression flag 'z' to the tar options, maybe then splitting is not needed at all.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .