5

I wrote a backup script a long time ago, which uses this part of the tar manual:

You may give multiple `--exclude' options.

--exclude-from=file

-X file

Causes tar to ignore files that match the patterns listed in file.

My tar command is the following:

includesFile=include.txt
excludesFile=exclude.txt
tar zcpf - . -T ${includesFile} -X ${excludesFile} | openssl des3 -salt | dd of=out.bak

This is include.txt:

/etc/
/var/
/usr/
/data/
/opt/
/root

This is exclude.txt

/data/webapp/webapp-data/*
/var/cache/*
/var/lib/dpkg/*
/usr/bin/*
/usr/share/locale/*

It used to work very well. Recently I noticed that the size of the archive increased significantly, and by switching to verbose tar made me see that it's including the webapp-data directory which is huge. For some reason it's not being excluded anymore.

The last thing I did is upgrading from Debian Jessie to Debian Stretch. The version change is minor, I wonder whether it's the reason.

I tried changing /data/webapp/webapp-data/* to /data/webapp/webapp-data/**, but that didn't help.

Why is the exclusion list not working anymore?

  • 2
    The manpage gives the create syntax as tar -c [-f ARCHIVE] [OPTIONS] [FILE...], I'm not sure the [OPTIONS] can come after the [FILE...] argument(s) like that - have you tried tar zcpf - -T ${includesFile} -X ${excludesFile} -- .? – steeldriver Aug 13 '17 at 17:43
  • @steeldriver I'll try that now and report. – The Quantum Physicist Aug 13 '17 at 17:44
  • 1
    @steeldriver Nope... didn't work either... – The Quantum Physicist Aug 13 '17 at 18:47
  • 1
    why use | dd of=out.bak rather than just > out.bak? BTW is /data/webapp/webapp-data/ the actual directory, or a symlink pointing to the directory? And is it failing to exclude anything in that exclude.txt file, or just the webapp-data directory? Also, try just /data/webapp/webapp-data/ without the * wildcards...IIRC that will exclude the directory and everything under it with tar. – cas Aug 14 '17 at 2:54
  • these lines includesFile=include.txt and excludesFile=exclude.txt doesn't contain any file content you set and passing to tar command!! – αғsнιη Aug 14 '17 at 6:09
3

I had the same problem with tar exclusions after upgrading from Debian Jessie to Stretch and I fixed it by just changing the commandline parameters order

from:
tar cvzpf backup.tar.gz /DirToBackup1 /DirToBackup2 --exclude-from=/path/to/backup_exclude.txt

to:
tar cvzpf backup.tar.gz --exclude-from=/path/to/backup_exclude.txt /DirToBackup1 /DirToBackup2

my backup_exclude.txt looks like this /var/log/* /cache/* /.cache/*

(refering to the end of the page of https://www.gnu.org/software/tar/manual/html_section/tar_49.html)

  • I didn't have any success with this... especially because I'm piping to openssl... I don't know where to put that - that takes care of piping... – The Quantum Physicist Sep 23 '17 at 5:08
1

I (finally) have figured out the problem and was able to solve it. It's a combination of multiple things, including that excludes are given as a wild-card (while before asking the question I didn't do that... now that doesn't work anymore). This is the command that works:

includesFile=include.txt
excludesFile=exclude.txt
tar -zcpf - --absolute-names -X ${excludesFile} -T ${includesFile} | openssl des3 -salt | dd of=out.bak

And definitely, excludes are given in this form:

/var/cache/*
/var/lib/dpkg/*
/usr/bin/*
/usr/share/locale/*
/proc/*
/sys/*

And includes:

/etc/
/var/
/usr/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.