2

I wrote a backup script on my Debian 8 system which uses tar command with "--exclude-from" to exclude some files/dir.

This works great, but today I would like to exclude some files sharing the same path pattern, like /home/www-data/sites/<some_string>log.txt or directories like /home/www-data/sites/<one_or_two_directories>/vendor.

I tried to append /home/www-data/sites/*log.txt into the file, but tar fails and outputs on stderr the following errors:

tar: /home/www-data/sites/*log.txt: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors

Did I miss something when trying to use * or ** ?

I then read that in Unix, programs usually do not interpret wildcards themselves which means that * isn't expanded neither ** by tar.

As far as I know, my last resort here is to expand the list using bash and append it into the exclusion file (if it's not already there) prior to the tar call. Is there a cleaner way?

EDIT

Here is the script snippet ..

# ...
broot=$(dirname "${PWD}")
i="${PWD}/list.include"
x="${PWD}/list.exclude"
o="$broot/archive.tgz"
tar -zpcf $o -T $i -X $x
# ...

Dans here is the exclusion file ..

/etc/php5/fpm
/etc/nginx
/etc/mysql
/home/me/websites/*log.txt
/home/me/websites/**/vendor

The goal is to exclude all log files located inside "websites" directory, and, all "vendor" directories that could be found in any subdirectories of "websites".

Thank you !

  • 1
    According to the info page for my version of GNU tar, --exclude-from will read patterns from that file, and the example is "*.o", so maybe your version of tar behaves differently? Are the exclude-file entries all one-per-line? – Jeff Schaller Dec 31 '15 at 15:57
  • 1
    Copy-paste the exact command you ran. I took a guess in my answer, but I may be off the mark, since my crystal ball didn't find your attempts. What were you trying to do with **? – Gilles Jan 1 '16 at 2:25
  • @JeffSchaller Yes, each path has its own line – Stphane Jan 2 '16 at 11:57
  • @Gilles I will post contents as an edit – Stphane Jan 2 '16 at 11:59
7

The shell expands wildcards in arguments, so most applications don't need to perform any wildcard expansion. However tar's exclude list does support wildcards, which happen to match the wildcards supported by traditional shells. Beware that there may be slight differences; for example tar doesn't distinguish * and ** like ksh, bash and zsh can. With tar, * can match any character including /, so for example */.svn excludes a file called .svn at any level of the hierarchy. You can use tar --no-wildcars-match-slash in which case * doesn't match directory separators.

For example, excluding /home/me/websites/*log.txt excludes /home/me/websites/log.txt, /home/me/websites/foo-log.txt and /home/me/websites/subdir/log.txt. Excluding /home/me/websites/**/vendor excludes /home/me/websites/one/vendor and /home/me/websites/one/two/vendor but not /home/me/websites/vendor. With the --no-wildcars-match-slash option, /home/me/websites/*log.txt does not exclude /home/me/websites/subdir/log.txt and /home/me/websites/**/vendor does not exclude /home/me/websites/one/two/vendor.

tar … --exclude='/home/www-data/sites/*include' … excludes the files and directories under /home/www-data/sites whose name ends with include. You might get away without the quotes, but not if you write --exclude /home/www-data/sites/*include (because then the shell would expand the wildcards before tar can see them) or if you use a shell that signals an error on non-matching wildcards (e.g. zsh in its default — and recommended — configuration).

The option --exclude-from requires a file name. The file must contain one pattern per line. Don't confuse --exclude (followed by a pattern) and --exclude-from (followed by the name of a file containing patterns).

  • It would be nice if OP quoted the actual command, but the comment about "append into" made it sound as if the --exclude-from option were used more/less correctly. – Thomas Dickey Jan 1 '16 at 12:01
  • it appears that --wildcars-match-slash option is default behaviour :/ When wildcards match slash (the default for exclusion members), … – Stphane Jan 2 '16 at 14:30
  • @Stphane Oh, right, tar is backwards, sorry. Fixed, and updated to match the question. – Gilles Jan 2 '16 at 14:38
  • @gilles thank you for the update .. Maybe I only needed AGILE assitance ^^ to point out some syntax error ^^ I will review all that, revise my scripts and post results. To be continued … – Stphane Jan 2 '16 at 14:52
0

Your command was probably improperly quoted (and a quick check here works as expected). According to the reference manual, GNU tar recognizes shell wildcards:

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.