I'd like to essentially tar/gz a directory on a remote machine and save the file to my local computer without having to connect back into my local machine from the remote one. Is there a way to do this over SSH? The tar file doesn't need to be stored on the remote machine, only on the local machine. Is this possible?
You can do it with an ssh command, just tell
tar to create the archive on its standard output:
ssh remote.example.com 'cd /path/to/directory && tar -cf - foo | gzip -9' >foo.tgz
Another approach, which is more convenient if you want to do a lot of file manipulations on the other machine but is overkill for a one-shot archive creation, is to mount the remote machine's filesystem with SSHFS (a FUSE filesystem). You should enable compression at the SSH level.
mkdir ~/net/remote.example.com sshfs -C remote.example.com:/ ~/net/remote.example.com tar -czf foo.tgz -C ~/net/remote.example.com/path/to/directory foo
Use this command to copy to a remote destination:
tar -c --zstd <src files> | ssh user@target_host "cd target_dir && tar -x --zstd"
-c: create new tar'd output (defaults to stdout)
--zstd: use zstd compression (faster than gzip, the algorithm for tar)
-x: extract output (defaults to stdin)
Measure line rate for benchmarking:
tar -c --zstd <src files> | pv --timer --rate | ssh user@target_host "cd target_dir && tar -x --zstd"
Environment: local wifi between laptop and desktop (limited to ~40MiB/s)