I'm looking for the fastest way of copying a file between 2 linux servers using bsdtar.

With normal tar, I would do something like:

ssh root@remote 'tar -cz -C /my/path/ big_file.txt' | tar -zxv

However, just replacing "tar" with "bsdtar" does not seem to work.

So, my questions are:

  1. How would I do this with bsdtar?
  2. Instead of using gzip, I would like to use something that can use multiple cores for the compress/uncompress, like pbzip2. How would I pipe this in?
  3. I want the file "big_file.txt" to be called "hello_world.txt" on the destination server without using a temp-file with renaming. How?


I solved number 1 myself:

ssh root@localhost 'bsdtar -cz -f - -C /my/path/ big_file.txt' | tar -zxv


I solved number 3 myself:

ssh root@localhost 'bsdtar -cz -f - -C /my/path/ big_file.txt' | tar -zxv -O > test.txt

There are many variations in the syntax of the tar command. GNU tar defaults to reading the archive on stdandard input and to writing the archive to standard output, but many other versions default to a tape device. Pass the f flag (with or without a dash — I think BSD versions support either) with the argument - to indicate stdin or stdout.

If you want to use a different (de)compression program, don't tell tar to compress, and pipe into the (de)compression program explicitly.

To rename the file, use the -s option, if your version of tar supports it (it is similar to the -s option of pax and to the --transform option of GNU tar).

ssh root@remote 'cd /my/path && bsdtar cf - -s '/big_file\.txt$/hello.txt/' big_file.txt | pbzip2' | pbzip2 -d | tar xf -

If you're only copying a single file, there's no point in using tar unless you both want to preserve metadata and want to use a custom compression program.

scp -C -p root@remote:/my/path/hello.txt big_file.txt

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.