Compress a directory using tar/gz over SSH to local computer?

Question

I'd like to essentially tar/gz a directory on a remote machine and save the file to my local computer without having to connect back into my local machine from the remote one. Is there a way to do this over SSH? The tar file doesn't need to be stored on the remote machine, only on the local machine. Is this possible?

Answer 1

You can do it with an ssh command, just tell tar to create the archive on its standard output:

ssh remote.example.com 'cd /path/to/directory && tar -cf - foo | gzip -9' >foo.tgz

Another approach, which is more convenient if you want to do a lot of file manipulations on the other machine but is overkill for a one-shot archive creation, is to mount the remote machine's filesystem with SSHFS (a FUSE filesystem). You should enable compression at the SSH level.

mkdir ~/net/remote.example.com
sshfs -C remote.example.com:/ ~/net/remote.example.com
tar -czf foo.tgz -C ~/net/remote.example.com/path/to/directory foo

Answer 2

Use this command to copy to a remote destination:

tar -c --zstd <src files> | ssh user@target_host "cd target_dir && tar -x --zstd"

Tar flags:

-c: create new tar'd output (defaults to stdout)

--zstd: use zstd compression (faster than gzip, the algorithm for tar)

-x: extract output (defaults to stdin)

Measure line rate for benchmarking:

tar -c --zstd <src files> | pv --timer --rate | ssh user@target_host "cd target_dir && tar -x --zstd"

Compression benchmark

Environment: local wifi between laptop and desktop (limited to ~40MiB/s)

Gzip: 0:04:59

Zstd: 0:03:04

Answer 3

For a simple way to copy a directory or file by compressing it only for the transport:

$ ssh domain.net 'ls foo'
file1   file2

$ ssh domain.net 'tar czf - foo' | tar xz

$ ls foo
file1   file2