Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have to download a file from this link. The file download is a zip file which I will have to unzip in the current folder.

Normally, I would download it first, then run the unzip command.

$ wget http://www.vim.org/scripts/download_script.php?src_id=11834 -O temp.zip
$ unzip temp.zip

But in this way, I need to execute two commands, wait for the completion of first one to execute the next one, also, I must know the name of the file temp.zip to give it to unzip.

Is it possible to redirect output of wget to unzip? Something like

$ unzip < `wget http://www.vim.org/scripts/download_script.php?src_id=11834`

But it didn't work.

bash: wget http://www.vim.org/scripts/download_script.php?src_id=11834 -O temp.zip: ambiguous redirect

Also, wget got executed twice, and downloaded the file twice.

Thanks

share|improve this question
    
In the latter example, wget probably was executed twice because the ? is a special character in the shell. Putting the URL in ""s should help. –  p-static Oct 8 '10 at 0:32
add comment

4 Answers

up vote 19 down vote accepted

You have to download your files to a temp file, because (quoting the unzip man page):

Archives read from standard input are not yet supported, except with funzip (and then only the first member of the archive can be extracted).

Just bring the commands together:

wget http://www.vim.org/scripts/download_script.php?src_id=11834 -O temp.zip; unzip temp.zip; rm temp.zip

But in order to make it more flexible you should probably put it into a script so you save some typing and in order to make sure you don't accidentally overwrite something you could use the tempfile command to create a safe filename for your temp file:

#!/bin/bash
TMPFILE=`tempfile`
PWD=`pwd`
wget "$1" -O $TMPFILE
unzip -d $PWD $TMPFILE
rm $TMPFILE
share|improve this answer
    
Great answer! I may use that script myself :-) –  Josh Dec 19 '10 at 21:54
add comment

I don't think you even want to bother piping wget's output into unzip.

From the wikipedia "ZIP (file format)" article:

A ZIP file is identified by the presence of a central directory located at the end of the file.

wget has to completely finish the download before unzip can do any work, so they run sequentially, not interwoven as one might think.

share|improve this answer
add comment

This thread seems to have a solution. Haven't tried it myself though.

http://serverfault.com/questions/26474/unzipping-files-that-are-flying-in-through-a-pipe

share|improve this answer
    
Welcome to Unix & Linux! Generally we like answers on the site to be able to stand on their own - Links are great, but if that link ever breaks the answer should have enough information to still be helpful. Please consider editing your answer to include more detail. See the FAQ for more info. –  slm Apr 25 '13 at 13:15
add comment

This is a repost of my answer to a similar question:

The ZIP file format includes a directory (index) at the end of the archive. This directory says where, within the archive each file is located and thus allows for quick, random access, without reading the entire archive.

This would appear to pose a problem when attempting to read a ZIP archive through a pipe, in that the index is not accessed until the very end and so individual members cannot be correctly extracted until after the file has been entirely read and is no longer available. As such it appears unsurprising that most ZIP decompressors simply fail when the archive is supplied through a pipe.

The directory at the end of the archive is not the only location where file meta information is stored in the archive. In addition, individual entries also include this information in a local file header, for redundancy purposes.

Although not every ZIP decompressor will use local file headers when the index is unavailable, the tar and cpio front ends to libarchive (a.k.a. bsdtar and bsdcpio) can and will do so when reading through a pipe, meaning that the following is possible:

wget -qO- http://example.org/file.zip | bsdtar -xvf-
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.