3

I've got a file.tar.gz that I create and backup to Amazon S3. It's gotten bigger than Amazon's limits and thus has not been backing up.

This is my current command:

tar -cpzf file.tar.gz directory/

Is it better, with TAR's --exclude functionality to skip a directory where the bulk of files live or TAR and Gzip the entire directory and then use something like 'split' to break it up into smaller chunks?

Improve this question
3
  • 1
    What about using things like duplicity?
    – Stéphane Chazelas
    Mar 10, 2014 at 21:01
  • 1
    What is the limit imposed? Maximum file size or maximum disk usage? What do you mean by "better"? In terms of what? Speed? Ease of use?
    – terdon ♦
    Mar 10, 2014 at 21:16
  • I've just started looking at duplicity. That may be a better solution than my homegrown script. The limit at Amazon is 4GB. My file, which is a backup of web uploads, has grown to over 6GB now. Incremental backups with something like duplicity may be a better strategy.
    – arsenicnet45
    Mar 10, 2014 at 21:18

2 Answers 2

Reset to default
1

If you are splitting, split according the directory structure. Much more portable and much easier to decompress. It's easy to break a multiply split file (it's all interdependent). On the other hand, if you have 10 archives instead of one, you can just untar all at once with globbing. No harder than one file.

Improve this answer
0

xz compresses even better than bzip2, but (again) not by very much.

You should look into just saving the files that have changed, or that aren't included e.g. in (re)installable packages.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.