19

As far as I know, WinRAR archives may contain ECC (error correction codes), so if the archive is slightly damaged, then it can be fixed by itself.

For example, I can first encode archives.tar to archives.tar.ecc, and then upload it to my server. If the file is slightly damaged after downloading by the client, then it can be fixed automatically without downloading the file again by decoding archives.tar.ecc. I think it will be a great idea if the network connection is unstable.

I wonder whether there is any (open-sourced) software run on Linux that can meet my needs.

Any suggestions?

Improve this question
1
  • to some degree, this already happens - each file's header chksum field is the sum of that header's constituent bytes - to include the size field. Many tars go further - GNU tar encodes sparse file holes in header metadata and upon extraction by any POSIX pax compatible archiver generates files containing information needed to rebuild the file, or by GNU tar to the original sparse file. It uses primarily encoded header variables definable as specified by POSIX for the pax utility w/ -o.
    – mikeserv
    Nov 30, 2014 at 8:51

5 Answers 5

Reset to default
8

I had this same problem and found that the solutions provided in the other answer were not acceptable since the par2 program/standard involves creating separate files to do recovery and verification. I, like the original asker I think, want a single file with error correcting codes.

I found the program rsbep which used to be in the package manager and then found the necessary code here: https://www.thanassis.space/rsbep.html

This allows one to create a file with Reed-Solomon error correcting codes and then retrieve the original file from the, possibly corrupted, ECC version.

Improve this answer
2
  • rsbep is great! But it's a bit different: it's an entire frozen OS, so you can add files inside but you cannot modify what is inside, and it's a bit more cumbersome to move around. But it's a really interesting approach, using FUSE, and opensource.
    – gaborous
    Dec 28, 2016 at 15:08
  • 1
    @gaborous, that's not true. There is a FUSE fs implementation using rsbep transparently, but that's only on top of the scripts that work on a file-by-file basis.
    – nyov
    Mar 25, 2019 at 14:01
8

You're interested on PAR2, which uses Reed-Solomon error correction. parchive is the ECC file format specification, for Linux you will want PyPar or par2tbb, and on Windows you use a GUI called QuickPar.

Improve this answer
2
  • 1
    The links for PyPar and par2tbb look broken to me. I found par2tbb here
    – Drist
    Mar 7, 2017 at 19:09
  • 3
    I think PyPar is completely unrelated to error correction.
    – mbarkhau
    May 20, 2018 at 18:11
3

If you want to have the ecc file separate from the archive, you can use pyFileFixity, an opensource Python project that aims to provide a next-gen PAR-like protection system against bit corruption.

If you want the ecc protection to be included directly in the archive, you can use the opensource DAR, a kind of next-gen TAR, which allows to generate both non-solid (ie, allows partial extraction of corrupted archives) and ecc (using PAR2) and catalog isolation (ie, save a backup of directory tree and filesystem meta-data) directly in the .dar archives.

Improve this answer
2

Another similar 2018 project is Redupe that seems to provide both separate and embedded file error correction in two separate tools:

The first tool, redupe , is modeled after compression tools like gzip or bzip2 , but adds redundancy instead of eliminating it.

The example in the webpage (changed for brevity) looks similar to your use case:

$ redupe home-backup.tar.gz
$ ls -l
-rw------- 1 rescrv 5625162218 home-backup.tar.gz
-rw-r--r-- 1 rescrv 6433996800 home-backup.tar.gz.rd

reundupe can then be used to restore the file and possibly correct small errors.

Improve this answer
1

You can use dvdisaster which works also for other file types than images. It is able to create an ECC file for an input file or use one to repair a file.

Improve this answer
3
  • This answer is not directly applicable to OPs issue.
    – number9
    Jun 27, 2022 at 15:52
  • 1
    dvdisaster can create ECC files for single files. You can use this ECC file to repair the original file. Of course, dvdisaster is usually used for other use cases but not limited to those.
    – user2525536
    Oct 11, 2022 at 21:05
  • That was not apparent in your initial response, you should edit it (as it was unclear, for example). Thanks
    – number9
    Oct 12, 2022 at 12:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .