2

I'm using tar like this,

tar -cjpvf a.tar "$pattern1" "$pattern2"

The problem is that pattern1 and pattern2 sometimes match the same file, but tar doesn't seem to filter these duplicates.

e.g The following example adds two duplicate entry in the archive

%> tar -cjpvf a.tar /etc/passwd /etc/passwd
%> tar -xvf a.tar
etc/passwd
etc/passwd

How can I filter them out?

1
  • 2
    Others are answering how to fix it (I would use xargs myself). For a quick why - remember that tar stands for tape archive, and is a format that can easily be appended to. So you could backup /etc/passwd today and then add /etc/passwd again to the same tape tomorrow. Mar 13, 2017 at 0:57

3 Answers 3

3

If pattern1 and pattern2 are shell wildcard patterns then you can use an or-pattern.

tar -cjpvf a.tar @($pattern1|$pattern2)

This requires ksh, bash or zsh. In bash, you need to run shopt -s extglob first to activate the @(…) pattern syntax. In zsh, you need to run setopt ksh_glob first (and also tell zsh that the variables contain patterns rather than strings, so @($~pattern1|$~pattern2)), or use the native syntax ($~pattern1|$~pattern2).

1

If you feel comfortable parsing the output of ls (and here assuming that none of the file names contain characters of $IFS or wildcard characters or start with -):

tar -cjpvf a.tar $(ls -d1 <pattern1> <pattern2> | sort | uniq)

or

tar -cjpvf a.tar $(ls -d1 <pattern1> <pattern2> | sort -u)

If you're not comfortable with parsing the output of ls, the right approach is to use find (here assuming GNU tar or compatible):

find -maxdepth 1 \( -name <pattern1> -o -name <pattern2> \) -print0 \
 | xargs -0 tar -cjpvf a.tar

(here assuming the list of files is small enough that only one invocation of tar is run. Also note that find doesn't omit hidden files by default).

3
  • sort has had a -u option for at least a decade now. Mar 13, 2017 at 0:54
  • Cool. And I'm glad you added xargs too, because find's -exec needs to die in a fire. ;) Mar 13, 2017 at 11:19
  • @Aaron, that's at least 3 decades as the sort in PWB unix in 1980 already had it. There aren't many good reasons of preferring xargs over -exec and many to prefer -exec over xargs. See also Why is looping over find's output bad practice? Mar 14, 2017 at 7:59
0

If the order matters. For instance, if you want to archive foo* and *.txt files, and you want the foo* files to appear first in the archive (but foo.txt not to be included twice). IOW, get foo.a foo.b foo.txt foo.z a.txt b.txt z.txt in that order:

With zsh:

files=(./foo*(N) ./*.txt(N))
(($#files)) && tar jcf file.tar.bz2 ${(u)files}

With tcsh:

set -f files = (./foo* ./*.txt) && tar jcf file.tar.gz2 $files:q

If you want to make sure the *.txt are last (foo.a foo.b foo.z a.txt b.txt foo.txt z.txt in that order):

With zsh:

files=(./foo*(N) ./*.txt(N))
(($#files)) && tar jcf file.tar.bz2 ${(uOa)${(Oa)files}}

(Oa being a parameter expansion flag that reverses the order of the array)

With tcsh:

set -l files = (./foo* ./*.txt) && tar jcf file.tar.gz2 $files:q

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .