6

I wrote a backup script on my Debian 8 system which uses tar command with "--exclude-from" to exclude some files/dir.

This works great, but today I would like to exclude some files sharing the same path pattern, like /home/www-data/sites/<some_string>log.txt or directories like /home/www-data/sites/<one_or_two_directories>/vendor.

I tried to append /home/www-data/sites/*log.txt into the file, but tar fails and outputs on stderr the following errors:

tar: /home/www-data/sites/*log.txt: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors

Did I miss something when trying to use * or ** ?

I then read that in Unix, programs usually do not interpret wildcards themselves which means that * isn't expanded neither ** by tar.

As far as I know, my last resort here is to expand the list using bash and append it into the exclusion file (if it's not already there) prior to the tar call. Is there a cleaner way?

EDIT

Here is the script snippet ..

# ...
broot=$(dirname "${PWD}")
i="${PWD}/list.include"
x="${PWD}/list.exclude"
o="$broot/archive.tgz"
tar -zpcf $o -T $i -X $x
# ...

Here is the exclusion file ..

/etc/php5/fpm
/etc/nginx
/etc/mysql
/home/me/websites/*log.txt
/home/me/websites/**/vendor

The goal is to exclude all log files located inside "websites" directory and all "vendor" directories that could be found within any sub-directories of "websites".

4
  • 1
    According to the info page for my version of GNU tar, --exclude-from will read patterns from that file, and the example is "*.o", so maybe your version of tar behaves differently? Are the exclude-file entries all one-per-line?
    – Jeff Schaller
    Commented Dec 31, 2015 at 15:57
  • 1
    Copy-paste the exact command you ran. I took a guess in my answer, but I may be off the mark, since my crystal ball didn't find your attempts. What were you trying to do with **? Commented Jan 1, 2016 at 2:25
  • @JeffSchaller Yes, each path has its own line
    – Stphane
    Commented Jan 2, 2016 at 11:57
  • @Gilles I will post contents as an edit
    – Stphane
    Commented Jan 2, 2016 at 11:59

2 Answers 2

12

The shell expands wildcards in arguments, so most applications don't need to perform any wildcard expansion. However tar's exclude list does support wildcards, which happen to match the wildcards supported by traditional shells. Beware that there may be slight differences; for example tar doesn't distinguish * and ** like ksh, bash and zsh can. With tar, * can match any character including /, so for example */.svn excludes a file called .svn at any level of the hierarchy. You can use tar --no-wildcards-match-slash in which case * doesn't match directory separators.

For example, excluding /home/me/websites/*log.txt excludes /home/me/websites/log.txt, /home/me/websites/foo-log.txt and /home/me/websites/subdir/log.txt. Excluding /home/me/websites/**/vendor excludes /home/me/websites/one/vendor and /home/me/websites/one/two/vendor but not /home/me/websites/vendor. With the --no-wildcards-match-slash option, /home/me/websites/*log.txt does not exclude /home/me/websites/subdir/log.txt and /home/me/websites/**/vendor does not exclude /home/me/websites/one/two/vendor.

tar … --exclude='/home/www-data/sites/*include' … excludes the files and directories under /home/www-data/sites whose name ends with include. You might get away without the quotes, but not if you write --exclude /home/www-data/sites/*include (because then the shell would expand the wildcards before tar can see them) or if you use a shell that signals an error on non-matching wildcards (e.g. zsh in its default — and recommended — configuration).

The option --exclude-from requires a file name. The file must contain one pattern per line. Do not confuse --exclude (followed by a pattern) and --exclude-from (followed by the name of a file containing patterns).

4
  • It would be nice if OP quoted the actual command, but the comment about "append into" made it sound as if the --exclude-from option were used more/less correctly. Commented Jan 1, 2016 at 12:01
  • it appears that --wildcars-match-slash option is default behaviour :/ When wildcards match slash (the default for exclusion members), …
    – Stphane
    Commented Jan 2, 2016 at 14:30
  • @Stphane Oh, right, tar is backwards, sorry. Fixed, and updated to match the question. Commented Jan 2, 2016 at 14:38
  • @gilles thank you for the update .. Maybe I only needed AGILE assitance ^^ to point out some syntax error ^^ I will review all that, revise my scripts and post results. To be continued …
    – Stphane
    Commented Jan 2, 2016 at 14:52
0

Your command was probably improperly quoted (and a quick check here works as expected). According to the reference manual, GNU tar recognizes shell wildcards:

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .