0

I encountered that the command bsdtar from the package libarchive (under Arch Linux, at least) does throw away executable bits of files in .zip-archives when reading from stdin, but not when directly working on the file.

On .tar-archives it preserves the executable bit also when reading from stdin.

Test case:

Create the archives:

Create the files:

touch a.txt
chmod 644 a.txt
touch a.out
chmod 755 a.out

The file permissions:

ls -ln a.out a.txt

shows

-rwxr-xr-x 1 1001 1001 0 Dec 12 11:01 a.out
-rw-r--r-- 1 1001 1001 0 Dec 12 11:01 a.txt

Pack the files into archives:

bsdtar --format=zip -cf a.zip a.out a.txt
bsdtar -cf a.tar a.out a.txt

(Creating the archives with zip and tar instead of bsdtar produces the same result.)

Extracting/ showing the archive content directly:

bsdtar -tvf a.zip

or

bsdtar -tvf - < a.zip

shows

-rwxr-xr-x  0 1001   1001        0 Dec 12 11:01 a.out
-rw-r--r--  0 1001   1001        0 Dec 12 11:01 a.txt

The executable bit of a.out is present here. The permissions of a.out are 755 and of a.txt 644.

Reading from stdin:

cat a.zip | bsdtar -tvf -

shows

-rw-rw-r--  0 1001   1001        0 Dec 12 11:01 a.out
-rw-rw-r--  0 1001   1001        0 Dec 12 11:01 a.txt

The executable bit for a.out is thrown away here. Furthermore, both files are group-writeable, they were not packed that way. The permissions of a.out and a.txt are both 664.

.tar-archive:

As a comparison, for a .tar-archive, the permissions in the archive are also honoured when reading from a pipe from stdin:

bsdtar --numeric-owner -tvf a.tar

and

cat a.tar | bsdtar --numeric-owner -tvf -

both show

-rwxr-xr-x  0 1001   1001        0 Dec 12 11:01 a.out
-rw-r--r--  0 1001   1001        0 Dec 12 11:01 a.txt

(note that, when showing the contents of a ZIP archive, bsdtar shows the numeric owner by default; for a TAR archive it shows the name of the owner.)

The question is:

What is special with stdin with regard to bsdtar? And why only when reading from a pipe, and not in the fashion bsdtar -tvf - < a.zip? And why special to a .zip-archive, but not to a .tar-archive?

  • i think the answer is in the source code. – Ipor Sircer Dec 11 '18 at 13:28
  • I'm mostly surprised that bsdtar read a Zip archive without complaining... – Kusalananda Dec 11 '18 at 15:01
  • @Kusalananda bsdtar (which is FreeBSD's tar) can both create and extract zip files (and a lot of other archive formats). – mosvy Dec 11 '18 at 15:18
0

Here on the bugtracker of libarchive is the answer:

Zip archives contains two different ways to describe the content:

  1. A per-entry header
  2. A central directory at the end of the zip file.

libarchive (and bsdtar by extension) will use the central directory if seeking is possible on the input, otherwise it will fall back to the streaming-only logic. The entries are not necessarily consistent as you found out in your test case. There isn't really much we can or want to do about this. Note that you can replace wget with a plain cat and it will still show the same behavior.

The short version is that this is an inherent issue with streaming of zip files and something that won't be fixed.

0

[this is not really an answer yet, but I'll post it as such since it's impossible to format anything in comments]

zip is not the only format which creates problems when extracted from a non-seekable file. Here is an example with multi-session iso images, but there at least bsdtar is printing an error message and is exiting with a non-zero status. It should probably do the same with zip files; silently messing up the permissions is not acceptable IMHO.

$ echo a.out > a.out
$ genisoimage -quiet -R -o a.iso a.out
$ chmod 755 a.out
$ growisofs -M a.iso -R -quiet a.out
Executing 'genisoimage -C 16,176 -M /dev/fd/3 -R -quiet a.out | builtin_dd of=a.iso obs=32k seek=11'
Rock Ridge signatures found
builtin_dd: 176*2KB out @ average infx1352KBps
a.iso: copying volume descriptor(s)

$ cat a.iso | bsdtar xf -
bsdtar: Ignoring out-of-order file @19 (a.out) 51200 < 411648
bsdtar: Error exit delayed from previous errors.
$ ls -l a.out; hd a.out
-rwxr-xr-x 1 ahq ahq 6 Dec 11 19:15 a.out
00000000  00 00 00 00 00 00                                 |......|
00000006

$ bsdtar xf a.iso
$ ls -l a.out; hd a.out
-rwxr-xr-x 1 ahq ahq 6 Dec 11 19:15 a.out
00000000  61 2e 6f 75 74 0a                                 |a.out.|
00000006
  • According to github.com/libarchive/libarchive/issues/…, it means libarchive creates ZIP archives with inconsistent metadata. – Golar Ramblar Dec 11 '18 at 16:02
  • I'd recommend editing this into the question as a useful way to recreate a test environment, actually. – Jeff Schaller Dec 11 '18 at 16:10
  • @GolarRamblar that's not my interpretation of it. bsdtar should create standard zip files by default; it's when extracting from a non-seekable file that it should process the central directory at the end and treat as an error when that contains any attributes that could've overridden those from the per-file header. – mosvy Dec 11 '18 at 17:37
  • @mosvy: Thanks for your better test case, I incorporated it into my question and removed it from your answer (edit still waits for approval). – Golar Ramblar Dec 12 '18 at 10:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.