116

Is there a way to add/update a file in a tar.gz archive? Basically, I have an archive which contains a file at /data/data/com.myapp.backup/./files/settings.txt and I'd like to pull that file from the archive (already done) and push it back into the archive once the edit has been done. How can I accomplish this? Is it problematic because of the . in the path?

1
  • 2
    +1. This would make working on OS X so easy. Currently, I have to hand clean a directory of all the hidden files like .DS_Store and then re-tar the directory.
    – user56041
    Jan 1, 2015 at 19:30

5 Answers 5

86

To pull your file from your archive, you can use tar xzf archive.tar.gz my/path/to/file.txt. Note that the directories in the file's path will be created as well. Use tar t (i.e. tar tzf archive.tar.gz) to list the files in the archive.

tar does not support "in-place" updating of files. However, you can add files to the end of an archive, even if they have the same path as a file already in the archive. In that case, both copies of the file will be in the archive, and the file added later will override the earlier one. The command to use for this is tar r (or tar u to only add files that are newer than the archive) is the command to use. The . in the path should not be a problem.

There is a catch, though: you can't add to a compressed archive. So you would have to do:

gunzip archive.tar.gz
tar rf archive.tar data/data/com.myapp.backup/./files/settings.txt
gzip archive.tar

Which is probably not what you want to hear, since it means rewriting the entire archive twice over. If it's not a very large archive, it might be better to untar the whole thing and then re-tar it after editing. Alternately, you could use an uncompressed archive.

7
  • 1
    This answer is only correct insofar as it deals with the standard tar program. Tar as a file format is actually very accessible. You could easily write a script to update the contents yourself, but there are other tools that already do this.
    – Caleb
    May 13, 2011 at 8:48
  • 30
    You could also do this the Unix way: gzip -dc archive.tar.gz | tar -r data/data/com.myapp.backup/./files/settings.txt | gzip >archive_new.tar.gz May 13, 2011 at 11:11
  • 2
    IIRC you can simply concatenate two gzip'ed files. The result is a larger gzip'ed file that contains both uncompressed files concatenated.
    – FUZxxl
    Mar 21, 2013 at 20:34
  • Once the file has been unzipped, isn't it possible to use the --update flag?
    – pietro
    Oct 6, 2017 at 10:13
  • 3
    tar: Options -Aru' are incompatible with -f -'
    – SineSwiper
    Jul 16, 2021 at 16:30
28

The tar file format is just a series of files concatenated together with a few headers. It's not a very complicated job to rip it apart, put your contents in, and put it back together. That being said, Jander described how tar as a program does not have the utility functions to do this and there are additional complications with compression, which has to both before and after making a change.

There are, however, tools for the job! There are at least two system out there which will allow you to to do a loopback mount of a compressed tar archive onto a folder, then make your changes in the file system. When you are done, unmount the folder and your compressed archive is ready to roll.

The one first option would be the archivemount project for FUSE. Here is a tutorial on that. Your system probably already has FUSE and if it doesn't your distribution should have an option for it.

The other option is tarfs. It's simpler to use, but I've heard it has some trouble with corrupting bzip2 archives so you might test that pretty thoroughly first.

4
  • 1
    I use AVFS, which is packaged in Ubuntu and Debian. Do you know how archivemount and tarfs compare to AVFS? May 13, 2011 at 11:31
  • 1
    Honestly no I don't, although I'm curious to look into AVFS now that you mention the alternative. I've used both archivemount and tarfs but it was for a long since decommissioned project and don't know their current state. I kind of decided from that experience that the idea was wrong conceptually even if the implementation worked perfectly.
    – Caleb
    May 13, 2011 at 20:28
  • 1
    I took a look at how archivemount works. It does not modify the existing .tar.gz file, but when you unmount, it renames the original and creates a brand new tar file in its place. If you're looking for a good user experience, it's great, but if you're looking to avoid rewriting the entire archive for efficiency reasons, it won't help you.
    – Jander
    Mar 16, 2016 at 18:24
  • @Jander That's for pointing that out. I'm pretty sure it applies to all the compressed archive mounting options though. The problem is the individual parts of the file are not compressed, the overall file internal headers and all is compressed and in order for other thing to extracti is as expected gzip has to be run over the whole data set. There isn't a way to inject something in the middle of this without extracting it and rerunning the compression and rewriting the whole data file. Hence my previous comment about it being the wrong solution conceptually. Archives are archives.
    – Caleb
    Mar 16, 2016 at 19:06
18

Tar was originally meant to be used for tapes, so "replacing" files is not really in the design. However, you can use --delete to delete the file from the original tar and then -u to update the tar and re-add the file. It may take a bit, as tar needs to reorganize the archive internally.

2
  • 1
    Welcome to StackExchange! Your answer only contains a suggestion already made in another answer. Please read the other answers to a question before answering, to avoid creating duplicates.
    – JigglyNaga
    Jul 29, 2016 at 19:56
  • 5
    I edited my answer a bit. I don't think the other answer mentions --delete
    – rleibman
    Jul 29, 2016 at 20:28
7

For tar.gz files, I'm afraid that this is not possible, according to my tests:

$ tar rvf backup.tar.gz db_bkp_today.sql 
tar: Cannot update compressed archives
tar: Error is not recoverable: exiting now

But being just Tape Archive (tar), I experimented positive results:

$ gzip -d backup.tar.gz
$ tar uvf backup.tar db_bkp_today.sql 
database_bkp_today.sql 
$ tar tvf backup.tar | grep sql
-rw-r--r-- ivanleoncz/ivanleoncz 14779319 2020-05-11 13:40 db_bkp_today.sql

Taken from man tar:

>  -r, --append
    Append files to the end of an archive.
    Arguments have the same meaning as for -c (--create).
>  -u, --update
    Append files which are newer than the corresponding copy in the archive. 
    Arguments have the same meaning as  with -c and -r options.

Using -u, has the same behavior of using -r, so the file was added to the TAR.

The practical way that I see for adding/updating a compressed Tape Archive (tar.gz), would ironically be:

  1. uncompress the tar.gz : gzip -d backup.tar.gz
  2. add/update a file on the tar file : tar uvf backup.tar db_bkp_today.sql
  3. compress the tar (with maximum compression if you want) : gzip -9 backup.tar
2
  • what's point of -r and -u options then if you have to unarchive and then create the archive again? From the man tar which you have added I understand that -u can be used if you want to replace an existing file present in the archive with the new version and so for the -r "Append files to the end of an archive". So for adding a new file can't -r be used? I could not understand what end of an archive meant there. Jun 24, 2021 at 12:44
  • 1
    @MohammadFaisal -r will add the file even if the same exists in the archive. -u will add the file only when it is not in archive yet or the file in archive is older. Both append the file to the end of the archive.
    – Marki555
    Feb 2, 2023 at 17:47
3

What you're looking for is zip. This is where zip shines above tar. The compression is really only slightly better with gzip or bzip2 when you're compressing a large directory structure with all sorts of files in it, i.e. making a running backup of a web server.

Testing on a typical WP install, tar.gz only saved about 4MB over zip in what amounted to about a 660MB archive file (663MB/zip) (559MB/gzip), bzip2 saved another 4MB (555MB), but is considerably slower and still lacks the ability to make incremental backups by simply running the same command to an existing archive.

zip --symlinks -r -FS /path_to_backup_dir/backup_file.zip /path_to_webroot/ -x /exclusions_as_pattern/

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .