6

I want to wget a file and tar it, in one command, I guess it is simple but I can't get it done. I tried several things.

wget <url> | tar -cvz file.gz.tar -

tar -cvzf file.tar `wget <url>`

wget -qO <url> | tar -cvf file.tar

wget <url> -O - | tar

Any help?

Improve this question
9
  • Your are downloading a single file or a sub-site (wget -r..)?
    – JJoao
    Commented Mar 24, 2015 at 9:47
  • 3
    Some sites let you wget -S --header="accept-encoding: gzip" url and obtain the gziped file; For a single file, file.gzip is a more natural solution than file.tgz.
    – JJoao
    Commented Mar 24, 2015 at 10:06
  • 5
    Tarring(?) a single file does nothing but waste bytes. Gzip OTOH may be what you want
    – Cole Tobin
    Commented Mar 24, 2015 at 14:55
  • 3
    Tar itself just merges multiple files into one file, which you don't want to do if you only have one file. Just use gzip to compress without tarring. There is no reason to use xxx.tar.gz instead of xxx.gz if you only have one file.
    – daboross
    Commented Mar 25, 2015 at 3:14
  • 1
    @giannis christofakis In fact I tried that, too but I seem to be not eligible to do so at the moment. However, accepting one of them clearly indicates that this is what you were looking for, not tar-ing, or at least that it is an accepatable alternative. Which is totally fine with me, but I think the question should reflect that.
    – inVader
    Commented Jan 12, 2018 at 9:49

4 Answers 4

Reset to default
19

Do you really want to tar the file or are you looking for downloading a file into a compressed form. Tarring a file is just bundling (uncompressed) files into an (uncompressed) archive. If you want to download a file into a compressed file you can use:

wget -qO - <url>|gzip -c - > file.gz

Improve this answer
8

You can't deal with input streams in that way. It is designed to deal with files. If you had managed to create an archive as you describe, what would it look like? How would you untar it? There would be no filename to create, just data.

I think your best bet is to get the file and tar it in two separate commands. If you don't want the file to remain on your drive, you can use tar's --remove-files flag:

wget http://example.com/filename && tar cvzf file.tgz --remove-files filename
Improve this answer
7
  • 2
    I'm sorry but I must disagree there tar is perfectly capable of dealing with input and output streams both reading and writing files, in fact my usual mode of operation for using the aur on Archlinux before I get cower installed is to use curl to download the tarball for it to the output stream type it into tar xz
    – hanetzer
    Commented Mar 24, 2015 at 14:42
  • 1
    @ntzrmtthihu777 yes but, presumably, you then redirect that to a file, you don't use the -f option to give a file name.
    – terdon ♦
    Commented Mar 24, 2015 at 14:44
  • nope. curl some://url/package.tar.gz | tar xf with no redirection
    – hanetzer
    Commented Mar 24, 2015 at 15:29
  • 3
    @ntzrmtthihu777 well yes, but that's totally different. You are receiving compressed data as input, and saving into a tarfile as output. The OP wants to receive text and pass that to tar. Try curl some://url/file.html | tar xf. That will fail.
    – terdon ♦
    Commented Mar 24, 2015 at 15:31
  • obviously. x is for decompression. Smart-aleky-ness aside, you are correct; I was misrecalling some experimentation I was doing with gzip. My prior point stands, however, that tar can read and write to stdout and stdin.
    – hanetzer
    Commented Mar 24, 2015 at 15:43
3

This is not going to work the way you want it to. A file (obviously with a filename) needs to be stored in the tar. That bit (the filename) is obviously missing if you just pipe the contents of the download to tar. I don't see any way to tell tar that it should pack stdin and specify a filename for that.

That said, I really do not see a way to achieve that with standard unix commands. Obviously, you might be able to write something yourself instead, for example using python and the request and tarfile modules which should not be to difficult if you are a bit handy with coding.

Until then, or if you have to rely on the available unix commands, you will need to stick to downloading to a temporary file, taring, then deleting the file. To not need to specify the filename multiple times, I would suggest using a variable:

DOWNLOAD=<FILENAME>; wget <DOMAIN>/$DOWNLOAD && tar -cvf $DOWNLOAD.tar --remove-files $DOWNLOAD
Improve this answer
1 
wget -qO - <url> |gzip -c > file_name.gz

-c for stdout . that's > used. the file get from wget serialize to file_name.gz using standard output library.

-qO to send output file

Improve this answer

Not the answer you're looking for? Browse other questions tagged .