Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have received a tar archive from a co-worker and after decompresing I found that some text files are corrupted. More precisely they are filled with zeros. They have correct size but all bytes are equal to 0x00.

Can this situation be caused by some uncompatibility of tar versions or eg. chinese character contained in the files or the files had to been corrupted in the time of compression? I do not expect that there was a problem during transfer because control checksum is OK.

share|improve this question
    
@goldilocks, 16-bit encodings contain plenty of zero bytes. (See my answer.) –  alexis Mar 4 '13 at 14:04
    
@alexis : Alright, I had thought UTF-16 always used the high bit as a marker for bytes 2-4, but evidently the BMP characters are a single 16-bit value and the top half of that would often be zero. Sorry! –  TAFKA 'goldilocks' Mar 4 '13 at 14:37
1  
Does tar tf the-archive.tar | sort | uniq -d report anything by any chance? (what about tar tvf the-archive.tar | grep -v '^[ld-]')? What the operating system and tar version and file system where the tar file was made and where it is extracted? –  Stephane Chazelas Mar 4 '13 at 15:36
    
The archive was created somewhere in China and I extracted in Czech Rep. in Ubuntu 12.10. Android sources was archived and some makefiles are broken as I described... –  Honza Mar 4 '13 at 18:18
add comment

2 Answers

up vote 0 down vote accepted

The most likely problem is that the tar got corrupted while it was being created. Due to the way the tar format is defined (since it is intended to be a streaming archiver) it must determine the file length ahead of time. It records this length in the tar header, then starts writing the file contents to the tar file. If for some reason there is an error reading the file, or if the file shrinks while it is being archived, it will fill in NULLs. This is needed so that the length specified in the header is still valid upon extraction (it can't go back and modify the header due to its streaming nature, and if it didn't pad the file with NULLs that would cause an error when extracting the next file in the archive).

Also, since tar deals with binary data (it has no "text" mode), there shouldn't be any issue (as far as tar is concerned) with different language encodings.

share|improve this answer
add comment

Are you sure all bytes are equal to 0x00? In that case your files contain no information at all (except for their size). No program could possibly store or transmit information as all zeros (unless it's telepathic).

What can happen is that a file has alternating text and zero bytes. Here's what this would mean: You've received files that contain unicode text, encoded as UTF-16 (or a near-equivalent). Each character takes up 16 bits (two bytes). Unicode assigns English letters and symbols to their ASCII character code, which means that, for example, letter A is hex 41 in ASCII and 00 41 in Unicode. The result is that if you write out "Hello" as UTF-16 and read it in as 8-bit text, you'll see this:

\0 H \0 e \0 l \0 l \0 o

In that case it wouldn't be the fault of tar. But if you really received all-zero files and the checksums check, there is definitely something wrong with the creating program. Not a version problem, but who knows? Perhaps a hardware problem causing the generating program to read all zeros.

(It's also possible, of course, that the files were correctly archived, and the bug is in the program that created the archived files).

share|improve this answer
    
Thank you for answer, but they are surely zero. There must be a problem on my coworker site... –  Honza Mar 4 '13 at 14:10
    
Ah, I'd say it definitely must be. Good luck with it then. –  alexis Mar 4 '13 at 14:12
1  
That's why it's called the Telepathic ARchiver ... or short TAR ... it ought to know! –  0xC0000022L Mar 4 '13 at 14:56
    
@0xC0000022L In that case I'd prefer it simply stored the number of zero bytes. Would cut down drastically on backup sizes. –  Michael Kjörling Mar 4 '13 at 15:08
1  
@MichaelKjörling Unix philosophy: tar produces long streams of 0 bytes, gzip transforms them into a count. –  Gilles Mar 4 '13 at 23:10
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.