101

I'd like to download, and extract an archive under a given directory. Here is how I've been doing it so far:

wget http://downloads.mysql.com/source/dbt2-0.37.50.3.tar.gz
tar zxf dbt2-0.37.50.3.tar.gz
mv dbt2-0.37.50.3 dbt2

I'd like instead to download and extract the archive on the fly, without having the tar.gz written to the disk. I think this is possible by piping the output of wget to tar, and giving tar a target, but in practice I don't know how to put the pieces together.

5 Answers 5

138

You can do it by telling wget to output its payload to stdout (with the flag -O-) and suppress its own output (with the flag -q):

wget -qO- your_link_here | tar xvz

To specify a target directory:

wget -qO- your_link_here | tar xvz -C /target/directory

If you happen to have GNU tar, you can also rename the output dir:

wget -qO- your_link_here | tar --transform 's/^dbt2-0.37.50.3/dbt2/' -xvz
5
  • To specified path should be: wget -qO- your_link_here | tar xvz - -C /target/directory
    – Marslo
    Sep 12, 2018 at 12:10
  • maybe just tell people to use tar instead wget then? May 19, 2019 at 4:29
  • 4
    wget -qO- <url> | tar -xvz -C <target folder> worked on gnu tar. Jun 22, 2019 at 14:22
  • will this require less than double the space of the archive on my disk? right now I'm having to unpack a huge tar and I'm looking for a way to optimize the operation space-wise
    – doplumi
    Jun 17, 2021 at 17:28
  • 1
    You don’t have to specify the stdin -. Jun 22, 2021 at 10:08
31

Another option is to use curl which writes to stdout by default:

curl -s some_url | tar xvz -C /tmp
4
  • 3
    I like your option more than others but curl -s some_url | tar xvz - -C /tmp
    – FiftiN
    Mar 18, 2019 at 17:44
  • 3
    as FiftiN suggested -> e.g. to view a filtered list of files inside repository one could use: $ curl -L https://api.github.com/repos/repo_owner/repo_name/tarball | tar tvfz - -C /tmp --wildcards *.py Apr 24, 2019 at 9:45
  • 6
    Better curl with "-L" to follow redirects
    – Rfraile
    Mar 27, 2020 at 15:37
  • works by default on a Mac too
    – ElFik
    May 11, 2021 at 22:03
13

This oneliner does the trick:

tar xvzf -C /tmp/ < <(wget -q -O - http://foo.com/myfile.tar.gz)

short explanation: the right side in the parenthesis is executed first (-q tells wget to do it quietly, -O - is used to write the output to stdout).

Then we create a named pipe using the process substitution operator from Bash <( to create a named pipe. This way we create a temporary file descriptor and then direct the contents of that descriptor to tar using the < file redirection operator.

1
  • This would need -f - (for stdin) or -f <(wget... to work. Nov 20, 2019 at 17:33
1

Named pipe with stdin solution and really mind the flags for tar's -xvz

tar -xvz -C /tmp/ -f <(wget -q -O - https://github.com/user/repo/release/download/v/v.tar.gz)
1

One liner that handles redirects and can extract tar.bz2 files. Use xzfor extracting gzip files.

curl -L https://downloads.getmonero.org/cli/linux64 | tar xj

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.