0

I'm setting up a server primarily as a NAS. It runs Ubuntu Server 22.04 LTS, the data is stored on a hardware controlled RAID.

Obviously I want to create a backup for off-site storage and used \

tar -cvpzf \
  /backup/location/backup1.tar.gz \
  --exclude=/some/files/* \
  /source/directory/

problem description:
Testing with a small portion of the data I got a successful backup. Upon trying the whole drive, the backup process stopped at some point and left me with a corrupted tar-file.

  • The point of failure occurred at different files. So far it seemed to happen when taring a .iso and a .exe file.
  • The size of the archive at that point was 5.4 GB
  • tar -tf /backup/location/backup1.tar.gz returns:
gzip: stdin: unexpected end of file
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now

Question 0: Would you use tar for this use case...
...or recommend another solution altogether?

Question 1: What is the reason for the archive breaking?
So far I'm guessing it may have to do with the archived filetypes or the overall size. Yet I didn't happen upon threads discussing this.

Question 2: How to avoid this crash?
Resulting from Q1, how can I backup without crash, ideally automatically checking the integrity during backup process?

Question 3: Can I repair the corrupted archive (excluding the last/corrupted file)?
Is there a way or do I need to recreate the backup once it's broken?

1 Answer 1

0

Testing with a small portion of the data I got a successful backup. Upon trying the whole drive, the backup process stopped at some point and left me with a corrupted tar-file.

You should probably investigate what failed. Reading the source files? writing the target file? Is that because the target file system is full? Or is it because either storage device is unreliable?

The good thing is that tar is an incredibly naive format. It's literally just one file description header, file contents, padding to next multiple of 512B, next file description header,…

So, all but the last file that were attempted to be stored are completely OK.

The bad thing is that tar is such an incredibly naive format. There's no checksums, only the length of the file, stored in the header, so that you can't know whether the file was written correctly.

  • Q0: I personally dislike tar; it's in many ways inefficient. I much prefer squashfs, because that actually includes a table of contents, so you don't have to read all the archive file to know what the name of the last file is – as in tar. It also comes with compression built-in, not applied afterwards, which again has advantages of being able to seek without having to decompress large swaths of the archive just to know where to seek to. It stores identical data only once. Finally, you can simply mount your squashfs archives as file systems, instead of having to extract it. Much more user-friendly.
    However, this is assuming you actually want to archive the full thing, every time, as archive. Usually you don't want that, but incremental backups. What tool you use depends on what your specific use case and target look like. There's actually a lot of discussion on how to backup on here; just to link to one solution which uses the snapshot functionality built into btrfs: Linux backup utility for incremental backups
  • Q1: we literally can't know. "stopped at some point" isn't enough description. You'll have to investigate this on your own a bit more!
  • Q2: see Q1.
  • Q3: nothing to repair: the archive up to the (excluding) last file is in perfect order. Simply extract from the archive.

By the way, you've got a modern system: don't use -z to compress when using tar; that's gzip, which is both relatively slow and relatively bad at compressing, compared to the also available --zstd.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .