0

Say I have a large tar, the contents of which I am regularly extracting to a constant location.

I am wanting to keep this location a pristine instance of the extracted tar, and I could avoid the very wasteful process of always wiping out the output directory just to update it with a small change the tar might have by using the --keep-newer-files flag.

But what would I do about removing files from the output location that no longer exist in the archive?

3

The standard tar command has a way to list the contents of an archive. The short form for this is tar -tf archive.tar.

By using this together with find, grep, and command substitution, you get the following:

rm -f $(find . -d | head -n -1 | grep -Fv "$(tar -tf archive.tar)")

Run this in the directory where you want the contents of the archive, and replace archive.tar with the path to the archive file.

Working from right to left for the pipeline inside the outermost command substitution:

  • find . -d: Lists all the files and directories under the current directory, listing them in depth-first order. The depth-first ordering is important here, as we plan to use this file list for deletions and therefore must delete directory contents before deleting directories themselves.
  • head -n -1: Exclude the last line from the find command. This will remove the entry for /path itself, which may or may not be present in the archive. If we weren't doing things in depth-first order, this would instead be tail -n -1.
  • grep -Fv "$(tar -tf archive.tar)": Pass the listing of the archive to a grep command, which will then filter out those lines from the list returned by find, so only the files which aren't' in the archive will be passed to the outter rm command.

A couple of notes:

  • This probably isn't quite POSIX compliant. It should work in GNU bash though.
  • This is dependent on how tar reports the archive contents and how the archive was created. In particular, the listing has to match up with how find reports things. With GNU tar and most archives, it should work. If it doesn't for your case, you can probably get it to work by using either sed or awk on the output of the tar command inside the innermost command substitution.
  • Make sure and test this before using it on production data, if it doesn't work for either of the two above reasons, it's liable to remove everything form the directory you run it in.
| improve this answer | |
  • Great answer. Some slight tweaks: 1. Find omits trailing slashes from directories, preventing matching those output from tar -t. 2. tar -t can potentially skip the printing of some directories. E.g. If a tar file contains only foo/dir1 and foo/dir2, tar -t will print those paths but not necessarily foo/. I can't explain this behaviour entirely either. I'm tolerating these directories not being removed for now - they will always be empty anyway. Thus I now have: rm $(find . -depth -type d -printf '%p/\n' -o -print | head -n -1 | grep -Fv "$(tar -tf ../*.tar)" | grep -v /$) – Brett Rogers Sep 24 '18 at 7:09
  • Why were my edits rejected @Goro? – Brett Rogers Sep 24 '18 at 8:51
  • @Goro and Romeo Ninov, small corrections to syntax errors to not fall under the "This edit was intended to address the author of the post" rejection reason. – Brett Rogers Sep 25 '18 at 5:06
  • @BrettRogers why do you think the find arguments are inverted? -d (= -depth) is a global option, so it can come before or after the .. – muru Oct 1 '18 at 2:14
  • @muru -d is indeed a global option, but is still considered part of the [expression] section, the entirety of which must be after the path argument. Trying to use -d before the path produces an error in GNU find. – Brett Rogers Oct 1 '18 at 4:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.