I am searching for a way to decompress a bzip2 compressed text file, change the content (adding new content, sorting etc.) and compress it again through pipelines.

I already found a way to do this but unfortunately I have to use another file as the recompressed output since bzip2 does not allow to use the same file in this case.

Here is my code:

bzip2 -dc file.bz2 | sort | bzip2 -9 > file_2.bz2

If I use the same file I get following error:

bzip2: Compressed file ends unexpectedly;
perhaps it is corrupted?  *Possible* reason follows.
bzip2: Success
Input file = file.bz2, output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

Any solutions how I can solve the issue?

Thanks in advance!

up vote 1 down vote accepted

Any solutions how I can solve the issue?

mv file_2.bz2 file.bz2

Also, you can use bunzip2 instead of bzip2 -d. The -c option to bzip2 is unnecessary in your use case.

Writing back compressed data in place is very tricky business because the compression factor varies through the file. So just don't go there.

In your case, sort is caching the entire file anyways. So it should be easy in principle, but there actually isn't an easy way to do it.

  • Okay great to know. I thought there is an easy way of doing in-place editing but it seems not. So I will take your suggestion :) Is there any noteworthy benifit of using bunzip2 instead of bzip2 -d? – linuxguy Dec 5 '16 at 0:57
  • @linuxguy readability. – DepressedDaniel Dec 5 '16 at 1:04

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.