Adding file to tbz files

Question

I'm looking for a way to update thousands of .tbz archive files, so I'll be doing this with a shell script. I need to add one file to each.

My question is, is there a faster way to do this without extracting each tbz's contents, then re-compressing with the new file included in the contained tar? What would the commands look like?

Thanks

Answer 1

While tar can add files to an already existing archive, it cannot be compressed. You will have to bunzip2 the compressed archive, leaving a standard tarball. You can then use tar's ability to add files to an existing archive, and then recompress with bzip2.

From the manual:

 -r      Like -c, but new entries are appended to the archive.  Note that this only
         works on uncompressed archives stored in regular files.  The -f option is
         required.

Answer 2

The other answer is correct: you cannot properly update a compressed tar archive without uncompressing it. The GNU tar documentation hints at it, and attempting to update fails with an explicit error message:

$ tar --concatenate --file=cat.tar.bz2 two.tar.bz2 
tar: Cannot update compressed archives
tar: Error is not recoverable: exiting now

However, should you be interested in a dirty sort-of-works solution that doesn't require decompression, I can provide one, based on the following observations:

• Appending bzip2 streams using cat is supported and produces a valid bzip2 stream (the same is true of gzip);
• appending tars using cat does not produce a valid tar file, which is why the --concatenate option exists, but we can ask tar to pretend it's valid:

It may seem more intuitive to you to want or try to use cat to concatenate two archives instead of using the --concatenate operation; after all, cat is the utility for combining files.

However, tar archives incorporate an end-of-file marker which must be removed if the concatenated archives are to be read properly as one archive. --concatenate removes the end-of-archive marker from the target archive before each new archive is appended. If you use cat to combine the archives, the result will not be a valid tar format archive. If you need to retrieve files from an archive that was added to using the cat utility, use the --ignore-zeros (-i) option.

Based on this knowledge, we can do, for example:

cat {one,two}.tar.bz2 >combined.tar.bz2

This results, as the documentation snippet above explains, in an invalid tar file, but using --ignore-zeros, it can still be read fully:

## Show contents of `one.tar.bz2'
$ tar tf one.tar.bz2
a
b

## Show contents of `two.tar.bz2'
$ tar tf two.tar.bz2
c

## Show contents of `combined.tar.bz2', bypassing the bad format
$ tar tif combined.tar.bz2
a
b
c

Note how the above lists all three files from the original two archives, whereas omitting -i (correctly) lists only the files from the first original archive:

$ tar tf combined.tar.bz2 
a
b

Once again, that's nothing more than a dirty trick, but it could be useful if you control both the writing and reading sides and can make sure that -i will be used when attempting to read from files created in this way.