12

Someone sent me a ZIP file containing files with Hebrew names (and created on Windows, not sure with which tool). I use LXDE on Debian Stretch. The Gnome archive manager manages to unzip the file, but the Hebrew characters are garbled. I think I'm getting UTF-8 octets extended into Unicode characters, e.g. I have a file whose name has four characters and a .doc suffic, and the characters are: 0x008E 0x0087 0x008E 0x0085 . Using the command-line unzip utility is even worse - it refuses to decompress altogether, complaining about an "Invalid or incomplete multibyte or wide character".

So, my questions are:

  • Is there another decompression utility that will decompress my files with the correct names?
  • Is there something wrong with the way the file was compressed, or is it just an incompatibility of ZIP implementations? Or even misfeature/bug of the Linux ZIP utilities?
  • What can I do to get the correct filenames after having decompressed using the garbled ones?
  • If you look up those bytes in the cp862 table does the file name match what you expect? Otherwise, do you know the native encoding of the source machine? – Michael Homer Dec 28 '15 at 20:11
  • Ditto for cp1255, and any other plausible encodings; it may be possible to just work it out based on what looks right. – Michael Homer Dec 28 '15 at 20:22
  • @MichaelHomer: No, it doesn't look like it matches. The native encoding of the source machine is whatever MS Windows uses when you set the regional settings to Hebrew-Israel, so I guess it's sometimes UTF-8 and sometimes CP1255. – einpoklum Dec 29 '15 at 21:41
12

It sounds like the filenames are encoded in one of Windows' proprietary codepages (CP862, 1255, etc).

  • Is there another decompression utility that will decompress my files with the correct names? I'm not aware of a zip utility that supports these code pages natively. 7z has some understanding of encodings, but I believe it has to be an encoding your system knows about more generally (you pick it by setting the LANG environment variable) and Windows codepages likely aren't among those.

    unzip -UU should work from the command line to create files with the correct bytes in their names (by disabling all Unicode support). That is probably the effect you got from GNOME's tool already. The encoding won't be right either way, but we can fix that below.

  • Is there something wrong with the way the file was compressed, or is it just an incompatibility of ZIP implementations? Or even misfeature/bug of the Linux ZIP utilities? The file you've been given was not created portably. That's not necessarily wrong for an internal use where the encoding is fixed and known in advance, although the format specification says that names are supposed to be either UTF-8 or cp437 and yours are neither. Even between Windows machines, using different codepages doesn't work out well, but non-Windows machines have no concept of those code pages to begin with. Most tools UTF-8 encode their filenames (which still isn't always enough to avoid problems).

  • What can I do to get the correct filenames after having decompressed using the garbled ones? If you can identify the encoding of the filenames, you can convert the bytes in the existing names into UTF-8 and move the existing files to the right name. The convmv tool essentially wraps up that process into a single command: convmv -f cp862 -t utf8 -r . will try to convert everything inside . from cp862 to UTF-8.

    Alternatively, you can use iconv and find to move everything to their correct names. Something like:

    find -mindepth 1 -exec sh -c 'mv "$1" "$(echo "$1" | iconv -f cp862 -t utf8)"' sh {} \;
    

    will find all the files underneath the current directory and try to convert the names into UTF-8.

    In either case, you can experiment with different encodings and try to find one that makes sense.


After you've fixed the encoding for you, if you want to send these files back in the other direction it's possible you'll have the same problem on the other end. In that case, you can reverse the process before zipping the files up with -UU, since it's likely to be very hard to fix on the Windows end.

  • I guess this will have to do since the ZIP file I was looking into is now gone for, well, reasons irrelevant here. Thanks, will do this next time and hope for the best. – einpoklum Dec 29 '15 at 21:49
  • 1
    rar or p7zip refuse to handle .zip archives. Is there a way to extract an archive with filenames in proprietary encodings, on Linux? When I extract with unzip, I get an error: "error: cannot create ╨и╨╕╨┐/Ship_╨п ╨Я╤А╨╛╤З╨╗╨░ ╨Я╤А╨╛ ╨н╤В╨╛ ╨▓╨Ю╨┤╨╜╨╛╨╣ ╨Ъ╨╜╨╕╨╢╨║╨╡!.png File name too long" – Nickolai Leschov Jan 13 '17 at 19:01
  • I managed to extract .zip file correctly with LANG=ru_RU.CP1251; unzip Bleed.zip (it was Cyrillic encoding in my case). Now I wonder how do I set up my system so that I can correctly open such .zip files in GUI by default? – Nickolai Leschov Jan 13 '17 at 19:18
  • @NickolaiLeschov Ask a question and someone may be able to help you. You'll probably need to provide more information about your system. – Michael Homer Jan 13 '17 at 22:09
  • unzip -UU foo.zip worked for Turkish characters – Mert S. Kaplan Nov 14 '17 at 20:04
6

I have just had the same problem, and it turns out that my version of unzip that is available from Ubuntu repositories (UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP.) can handle automatic decoding of filenames if you specify the -a switch.

unzip -a stupid.zip
  • +1 although I have nothing to test this with right now. – einpoklum Mar 4 '18 at 15:32
  • 1
    According to the man page of unzip the -a switch takes care of converting text files. Not file names. – beruic May 7 '18 at 14:04
  • @beruic, I had unzipped unicode numbers as file names (#U+0040#U+0050...), and then "unzip -a" really helped. – chang zhao Jan 20 at 6:07
  • Thanks @Socowi, removed the unneeded switch. – Igor Zinov'yev Apr 22 at 12:05
4

I had success with the command 7z x <source.zip>.

Version:

p7zip Version 16.02 (locale=utf8,Utf16=on,HugeFiles=on,64 bits,[...])

Potentially relevant environment:

LANG=en_US.UTF-8
LC_ALL=en_US.UTF-8
LC_CTYPE=UTF-8

It was able to decompress all files with 8-bit characters in their filenames, with some of these characters skipped, some garbled.

  • p7zip is the only one that worked for me – alex88 Mar 14 at 14:19
2

I had luck with this combination:

export LANG=es_MX 
7z x file.zip
convmv -f cp437 -t utf8 -r .

add --notest to convmv for actual rename. Later I found even better version:

LANG=es_MX.cp437 unzip -UU file.zip
convmv -f cp437 -t utf8 -r . --notest
  • +1 just for convmv and the command-line 7zip tool. – einpoklum Feb 19 at 10:37
  • Probably one has to try different options for LANG variable and from encoding depending on the file at hand. I've got a file where LANG=ru_RU.CP1251 together with -f cp866 worked. – Dmitri Chubarov Apr 29 at 7:48
1

I had a similar problem with decoding a zip archive with cyrillic characters. A one line python script did the job properly:

#!/usr/bin/python

import zipfile
import sys

zipfile.ZipFile(sys.argv[1], 'r').extractall(sys.argv[2] if len(sys.argv) > 2 else '.')

Then just call it unzip_enc and call it unzip_enc ZIP_FILE [TARGET_DIR]

For me neither the unzip -UU, unzip -a nor LANG* environment variables did any good.

  • I'll try that next time I have such a zip file to extract... thanks. But - could you change your script so as to: 1. check whether there are two arguments 2. extract to the current working directory if only the zip file is provided? – einpoklum Feb 2 at 17:36
0

I have zip archive compressed in Linux (from command line) and filenames with diacritics characters are not correctly decompressed on Windows, but I succesfully unpacked it with Bandizip software which can set charset on toolbar.

protected by Community Apr 22 at 12:21

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.