I have a tar.gz file that when uncompressed will use 2 TB of data. On my machine I have 1.5 TB free on one drive and 1.2 TB free on another. I would like to untar this large archive over these two drives. I don't mind how the data is split as long as the data is present on these 2 drives.

Is this possible?

The top level directories in the tar archive are numerically numbered from 00000 to 99999.

  • Are the data evenly distributed between the directories? – ErikF Jan 25 '18 at 23:49

To split it by halves, based on your given directory names and presuming that your data are evenly distributed therein:

tar -C /path/to/mount1 xf /path/to/tarball.tgz {00000..49999}
tar -C /path/to/mount2 xf /path/to/tarball.tgz {50000..99999}

Just as when creating tarballs, you give it a list of files to archive; when unrolling tarballs, you give it a list of files to extract. -C tells tar what the working directory of the extract operation should be.

  • So just to clarify, {00000..49999} would really mean actual filenames or file name patterns that would match about half of the files? In practice, the OP could do a tar tvf tarball.tgz and look for which files to place where... and some trial and error. – Mark Stewart Jan 25 '18 at 15:41
  • This looks really good. Will this work with wildcards e.g. {0*..4*}? Will it also work if I had archives where there was a path in front of it, e.g. {./some/nest/0*.../some/next/4*? – CadentOrange Jan 25 '18 at 15:58
  • {0..9} would expand to 0 1 2 3 4 5 6 7 8 9 before being handed to tar. You cannot use wildcards in brace expansions, so far as I am aware. However, you can use wildcards in the paths to the actual tarballs, and in the lists of files to extract. That said, tar f expects one file name to be provided which would be the tarball upon to perform the operation. To work on multiple tarballs, you'd want to perhaps iterate a for loop across the results of a find that returns the tarballs in question. – DopeGhoti Jan 25 '18 at 16:02
  • Less onerous, however is wildcards in the target filenames, e. g. tar xf /path/to/tarball.tgz {00000.49999}/batch* would only catch files and subdirectories matching the glob batch* within the numerical top-level directories. – DopeGhoti Jan 25 '18 at 16:04
  • 2
    Some tar commands have a --wildcards option to specify patterns of file to extract. – Patrick Mevzek Jan 25 '18 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.