4

I want to extract few directories from very big tar.gz archive (lets assume aaa,bbb,ccc). ccc sometimes exists, sometimes not, but I want to avoid fails during extract process.

Is it possible to extract exact list of directories from tar archive and not fail if some of them are not present?

Command like tar -xzf file.tar.gx --wildcards aaa bbb ccc fails if any position is not present in archive.

2
  • 1
    Could you not just ignore the error? Commented Nov 27, 2022 at 20:58
  • Your suggested edit came up in review, and I had to reject it as it didn't make sense in the context of Stephane's answer. You could post it as a comment, or edit your question with something along the lines "I tried Stephane's suggestion but my case is even more complicated because...". Commented Apr 1, 2023 at 15:03

3 Answers 3

4

One option could be to use pax, the standard command to extract tar archives and use the -'s/regexp/replacement/ option to remove the members we don't want from the selection:

<file.tar.gz gunzip |
  pax '-s:'{aaa,bbb,ccc}':&:' \
      '-s:.*::' -r

The idea being that for all archive members that contain either aaa, bbb or ccc in their path, we replace those with the same, so a no-op but that results in the next substitutions to be skipped when there's a match and in particular the s:.*:: which has the effect of discarding the archive member.

To see what would be extracted, you can remove the -r. Add the p flag to the substitutions so get a report of what substitutions are being made.

Beware that with the pax implementation from MirBSD as found on Debian/Ubuntu at least, I find that if there are archive members that are symlinks, they are discarded if the symlink target doesn't match the patterns (even if their path does). See https://austingroupbugs.net/view.php?id=1618 for where improving the API in this regard is currently being discussed.

5
  • hah, nice move! Commented Nov 27, 2022 at 20:33
  • 2
    I don't think pax is the standard command to extract tar archives? I think that's tar Commented Nov 28, 2022 at 17:02
  • 1
    @user253751, in POSIX (now merged with the Single UNIX Specification), tar and cpio were obsoleted long ago and replaced with pax. The last version to have tar (marked legacy) was SUSv2 (1997). It's true it didn't get a wide following though. Commented Nov 28, 2022 at 17:29
  • 1
    I suppose the good thing about standards is there are so many to choose from! Commented Nov 28, 2022 at 17:36
  • @user253751, there aren't the many standards (that I know at least) that broadly cover operating system interface (OSI in POSIX) especially command line interface. Most of the ones I know are built upon POSIX (like the Debian policy or LSB). Commented Nov 28, 2022 at 19:56
3

The GNU and BSD tar commands don't support that as far as I know, but:

If the tar archive is small enough or not read from a medium that's expensive to rewind (actual tape archives), you could list_of_matching_files=$(tar -tf file.tar.gz | grep '(aaa|bbb|ccc)') to compile a list of files. You will not like what happens when file names contain line breaks, which is perfectly legal.

So, that would leave you (at least in GNU tar) with the option to execute a command on each executed file, using the --to-command= option. tar will set the TAR_REALNAME environment variable, which your program can use to select whether the data piped in will be written to a file of an appropriate name, or just ignored. You should then also handle the other TAR_** environment variables set to handle file/directory types, owners, modes and dates correctly. In short, aside from reading the (pretty stupid) .tar format, you'd be doing tar's job in a program / shell script of your own.

Alternatively, honestly, since a tar needs to be read sequentially anyways, and storage is usually cheap, just extract everything, noting the extracted files on the way, and delete the "wrong" ones later.

Even more alternatively, might be worth trying whether 7z extracting a tar file also aborts when a pattern doesn't match.

And, finally: Every proper programming language probably has a tar-consuming library. Might really be worth the six-ish lines of Python, see the second example from the official documentation:

#!/usr/bin/env python3
import os
import tarfile

def py_files(members):
    for tarinfo in members:
        """
        modify this check: only `yield tarinfo` if the
        tarinfo.name matches your needs. Conveniently,
        python has string functions like `tarinfo.name.startswith("foo")`
        and a capable regex library
        """
        if os.path.splitext(tarinfo.name)[1] == ".py":
            yield tarinfo

tar = tarfile.open("sample.tar.gz")
tar.extractall(members=py_files(tar))
tar.close()
0

Regardless of the error, tar will extract the other files. If you don't care about the error, ignore it. Send stderr to /dev/null and don't test the return code $?.

tar -xzf file.tar.gz files 2> /dev/null

If you need to run commands sequentially, use ; instead of &&.

tar ... ; ...

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .