2

I'm rewriting my original question because I realized this is more efficient:

I'd like to tar and gzip my website files and then download it.

I know I can tar and gzip it with one command using tar with cvzf.

And then I can download it to my personal computer using rsync with a command like: rsync -av --stats user@remote.machine:/remote_directory_path/filename /local_directory_path/filename

Can I combine this into one command?

Or if someone can suggest their best methods for downloading a web directory to their local machine for backup purposes as one file.

  • Do you particularly want a local copy of the full directory tree on your local computer, or is that just a means to getting a tar.gz backup of the remote server? – roaima Jan 26 at 17:38
  • roaima, I just want to backup my website files and download it in one command. But, I want the files in one compressed file. I re-wrote my question after thinking about your comment. – linstar Jan 26 at 19:23
3

You say that you don't really want the remote server's directory tree mirrored onto your local machine. In that case rsync is the wrong tool. You could tar the remote content directly, delivering the compressed tarball to your local machine:

ssh user@remote.machine tar -C /path/to/directory -czf - . | cat >tarball.tgz

Unlike rsync, this maintains no state locally, so if the connection drops during transfer you will have to transfer the entire archive again. One way around this would be to tar the content on the remote server and then use rsync to transfer that:

ssh user@remote.machine 'tar -C /path/to/directory -cf - . | gzip --rsyncable >tarball.tgz'
rsync -av user@remote.machine tarball.tgz .

The --rsyncable flag provides restart markers inside the compressed file so that small changes in the uncompressed original don't change the entire compressed result.

  • This looks like what I want. I'll have to try it and let you know. Thanks. – linstar Jan 26 at 19:37
  • Worked, thanks! Added some -v options to see what's going on too. Just a question though: I logged into my webhost to check if it left the file there. It didn't, which is great because I thought I may have to modify this command to delete the file too. But, it wasn't there. Can someone go over this whole command to explain why the compressed file doesn't remain on my webhost? – linstar Jan 27 at 14:55
  • I used the 2nd example. Was it supposed to leave the file if I used your 2nd example? – linstar Jan 27 at 16:12
  • I couldn't find the file. I've tried it twice now. Anyways, it's working the way I want because I don't want the file left there anyways. Also I modified the command this way so that the tar'd path is in the compressed file: ssh user@remote.machine tar -cvf - /path_to_source_files . | gzip --rsyncable >tarball.tgz rsync -av user@remote.machine tarball.tgz . – linstar Jan 27 at 16:54
  • Interesting, so just by leaving out the quotes, the file gets deleted. – linstar Jan 28 at 3:28
0

You can compress the files during transfer with the -z option, but this has no effect on your received files (other than a faster transfer), i.e. they are not "gzipped" into a single file.

You could tar the remote directory and then rsync a single file. This can speedup the transfer process if you have lots of small files to copy.

But it also makes sense to rsync the directory and build a tar afterwards (as backup file), just as you did. The next time you do this, only changed files will be transferred, hence the name rsync.

  • The transfer would only be faster with -z if the source and destination machines were able to compress and decompress data quicker than what the network's bandwidth is. – Kusalananda Jan 26 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.