7

There are a few questions here about non-ASCII letters in the names of files stored as streams inside zip files (Hebrew, Chinese, Japanese or Korean). However none of the solutions provided helped me with a zipfile with Cyrillic letters that came from a Windows machine.

The file has a cyrillic name itself (Космос.zip - downloadable link). This is an archive with zero-length contents just for the purpose of illustration.

unzip -l prints:

Archive:  Космос.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2017-05-03 18:19   ɫ���߼��/ict_inf.pdf
---------                     -------
        0                     1 file

The ugly ɫ���߼�� stands for the sequence of bytes C9 AB DF E8 AB DF BC AB DF.

I know (by using GMail preview feature) that this should be

Archive:  Космос.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2017-05-03 18:19   РосКосмос/ict_inf.pdf
---------                     -------
        0                     1 file

That is we need to map C9 AB DF E8 AB DF BC AB DF to РосКосмос.

There are several commonly used 8-bit cyrillic encodings: CP1251, CP866, ISO8859-5, however they would have this word encoded as a different sequence of bytes:

           Р  о  с  К  о  с  м  о  с
CP866:     90 AE E1 8A AE E1 AC AE E1
CP1251:    D0 EE F1 CA EE F1 EC EE F1
ISO8859-5: C0 DE E1 BA DE E1 DC DE E1

Clearly none of the commonly used 8-bit cyrillic encodings would decode the input names to the output names like this. There is something more complicated at work here.

If only we knew how to decode the names, renaming the files after extraction would be easy with an appropriate find script (https://unix.stackexchange.com/a/252000/17649), e.g.

find -mindepth 1 -exec sh -c 'mv "$1" "$(echo "$1" | here-goes-the-decoding pipeline )"' sh {} \;

or the convmv utility.

1

Your ZIP file used with a "recent" infozip displays the right filenames:

unzip -l Russian-Космос.zip 
Archive:  Russian-Космос.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  2017-05-03 18:19   РосКосмос/ict_inf.pdf
---------                     -------
        0                     1 file

And unzip correctly creates the РосКосмос/ directory when unzipping.

UTF-8 support has been added to infozip long ago. Executables on my Ubuntu:

UnZip 6.00, 20 April 2009
Zip 3.0,  July 5th 2008

So your problem may be an ancient InfoZip version (or a version compiled without UTF-8 support)

In my version, strings /usr/bin/unzip | grep -A8 -B8 'UTF-8' yields, among other things:

ZIP64_SUPPORT (archives using Zip64 for large files supported)
LARGE_FILE_SUPPORT (large files over 2 GiB supported)
other
UTF-8
UNICODE_SUPPORT [wide-chars, char coding: %s] (handle UTF-8 paths)
USE_DEFLATE64 (PKZIP 4.x Deflate64(tm) supported)
USE_UNSHRINK (PKZIP/Zip 1.x unshrinking method supported)

which seems to be related to compile/build options

  • Interesting, it does not work for me. I am using the same Info-ZIP UnZip 6.00 but from OpenSUSE distribution. Will investigate. – Dmitri Chubarov Aug 20 at 7:42
  • UTF-8 support is a compile-time option, possibly not used in your distro. See my edited answer. – xenoid Aug 20 at 8:24
  • I did not test it but my impression is that the issue is caused by the unzip-iso8859_2.patch that is applied to OpenSUSE packaged unzip since Jan 24, 2002. Will probably file a bug report. – Dmitri Chubarov Aug 20 at 15:45
  • 1
    Shouldn't be too hard to recompile your own... – xenoid Aug 20 at 19:12
1

I have found a solution on the OpenNET.ru forum, a popular Russian-language resource that is dedicated to open-source software and technologies since 1996. A post on OpenNET suggests that Info-ZIP, once a popular set of tools for handling ZIP archives on computers running MS-DOS assumed that on MS-DOS there is only one 8-bit encoding, namely CP850, therefore all filenames are automatically run through CP850->CP1252 conversion. CP1252 was probably chosen as the most popular approximation of the ISO-8859-1 character set encoding.

Therefore the correct find command to run after extracting an archive containing Cyrillic filenames would be

find -mindepth 1 -exec sh -c 'mv "$1" "$(echo "$1" | iconv -f cp1252 -t cp850 | iconv -f cp866 )"' sh {} \;

Interestingly one can find suggestions to use not CP1252 but ISO-8859-1. This does not seem to be the case as one some of the archives that I have encountered the transformation iconv -f iso8859-1 -t cp850 failed while iconv -f cp1252 -t cp850 converted successfully.

Getting back to individual characters

           Р  о  с  К  о  с  м  о  с
CP866:     90 AE E1 8A AE E1 AC AE E1

Now applying CP850 -> CP1252 results in C9 AB DF E8 AB DF BC AB DF. Exactly the sequence that we have observed.

Another useful command would be

 unzip -l РосКосмос.zip | grep -aEv '^Archive:' | iconv -f iso8859-1 -t cp850 | iconv -f cp866

To get a list of files from the archive

 Length      Date    Time    Name
---------  ---------- -----   ----
        0  2017-05-03 18:19   РосКосмос/ict_inf.pdf
---------                     -------
        0                     1 file

Filtering away the line that starts with Archive: is a protection to hide the name of the archive from conversion.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.