Episode #125 of the Stack Overflow podcast is here. We talk Tilde Club and mechanical keyboards. Listen now

Hot answers tagged

163

Advantages of using .tar.gz instead of .gz are that tar stores more meta-data (UNIX permissions etc.) than gzip. the setup can more easily be expanded to store multiple files .tar.gz files are very common, only-gzipped files may puzzle some users. (cf. MelBurslans comment) The overhead of using tar is also very small. If not really needed, I still do not ...


74

You will need to join them first. You may use the common linux app, cat as in the example below: cat test.zip* > ~/test.zip This will concatenate all of your test.zip.001, test.zip.002, etc files into one larger, test.zip file. Once you have that single file, you may run unzip test.zip "How to create, split, join and extract zip archives in Linux" may ...


63

You are actually asking only half of the question. The other question being, "Why would I compress a tar file with gzip?". And the answer is not just that gzip makes the file smaller (in most cases): tar: stores filename and other metadata: mode, owner ID, group ID, filesize, modification time stores a checksum (for the header only) gzip: can store the ...


38

The Linux unzip utility doesn't really support multipart zips. From the manual: Multi-part archives are not yet supported, except in conjunction with zip. (All parts must be concatenated together in order, and then zip -F (for zip 2.x) or zip -FF (for zip 3.x) must be performed on the concatenated archive in order to “fix” it. Also, zip 3.0 and ...


30

There is a quite big advantage to using only-gzipped text files - the contents can be directly accessed with command-line tools like less, zgrep, zcat.


24

7z x archive.zip.001 It will automatically find the rest


23

You should check if most of your time are being spent on CPU or in I/O. Either way, there are ways to improve it: A: don't compress You didn't mention "compression" in your list of requirements so try dropping the "z" from your arguments list: tar cf. This might be speed up things a bit. There are other techniques to speed-up the process, like using "-N " ...


21

I would say it's likely that the people just don't realise they can use gzip/bzip2/xz without tar. Possibly because they come from a DOS/Windows background where it is normal for compression and archiving to be integrated in a single format (ZIP, RAR, etc). While there may be slight advantages to using tar in some situations due to the storage of metadata ...


20

I found the answer here: https://superuser.com/a/517758/10264 This answer is similar in concept to that of Gilles, namely first you combine the split archive into a normal archive using split, and then you unpack it using unzip. The difference is that instead of using the -FF flag, which did not work for me, you just tell zip to repack the split file ...


20

You can list the content of the archive and then pass the list to rm using xargs Example for a tarball (test it without the rm first): tar tfz archive.tar.gz | xargs rm -rf


18

I first want to clarify that of the list you provided, tar is the only one that is not a compression algorithm. tar is short for Tape Archive, and is used to create archive files. In short, a single file that consists of one or more files. In terms of availability, 7zip is widely available across UNIX (Linux/BSD/MacOS) and Windows systems. Therefore a zip ...


18

The usual tar format stores timestamps as a number of seconds, as you determined. To store timestamps with more resolution, you need to use another format, such as POSIX tar as produced by GNU tar which stores timestamps in nanoseconds (usually). To create these archives, use the -H posix option with GNU tar: tar cfH archive.tar posix ...


17

There is an important difference that could make using tar important under some circumstances: Besides the "metadata" that @jofel mentioned in his answer, tar records the filename in the archive. When you extract it, you get the original filename regardless of what the archive is called. In your case the tar archive and the file it contains have the related ...


16

It's probably a GNU specific option, but you could use the -O or --to-stdout to extract files to standard output ~/test$ mkdir foo ~/test$ cd foo ~/test/foo$ cat >bar hello world ~/test/foo$ cd .. ~/test$ tar -acvf file.tgz foo foo/ foo/bar ~/test$ ls file.tgz foo ~/test$ rm -rf foo ~/test$ tar -axf file.tgz foo/bar -O hello world ~/test$ ls file.tgz


15

The version of tar on OpenWRT is a smaller one than the one on full-blown systems, designed to fit small devices (it's BusyBox.) To keep small, it lacks features such as the automatic detection of compressed archives. Try declaring the compression format manually with the -z option: tar -xvzf ejdk-8u65-linux-arm-sflt.tar.gz Support for gzip in the tar ...


15

It's important to understand there's a trade-off here. tar means tape archiver. On a tape, you do mostly sequential reading and writing. Tapes are rarely used nowadays, but tar is still used for its ability to read and write its data as a stream. You can do: tar cf - files | gzip | ssh host 'cd dest && gunzip | tar xf -' You can't do that with ...


14

The OP asked about parallel archiving, not parallel compression. If the source material is coming from a filesystem where different directories/files might be on different disks, or even a single fast disk that exceeds the input speed of the compression tool(s), then could indeed be beneficial to have multiple streams of input going into the compression ...


12

gzip works with files, not directories. Therefore to create an archive of a directory you need to use tar (which will create a single tarball out of multiple files): tar cvf myetc.tar /etc or, for a gzipped archive: tar cvzf myetc.tar.gz /etc


12

On the line $ tar cvzf file.txt file.tar.gz you have asked to create a compressed archive of the file file.tar.gz, and that the archive should be called file.txt. You have simply swapped the two filenames around. The file file.txt will be overwritten by an empty compressed tar archive. This is why the subsequent commands displays weird things. Yes, ...


11

tar --use-compress-program=pigz .... replace pigz with your favorite parallel compression program. The reason to use tar is because it can store the owner, group, permissions. That metadata is often useful (e.g., restoring a dir tree in a complex system).


11

No, dd can be not sufficient. As examples: If you have a multi-track cd-rom with data and audio track mixed, using dd you will copy only the first data session. Many old video-games cd (on Play-Station 1) use as copy-protection some fake session on the cd. You have to replicate them to obtain a working cd. I successfully used cdparanoia to backup quite all ...


11

filename=/tmp/foo.gz if 7z t $filename; then 7z e $filename else echo $filename not an archive. fi


11

When searching for a single file in a large archive, it uses method 1, which you can see using strace: open("dataset.zip", O_RDONLY) = 3 ioctl(1, TIOCGWINSZ, 0x7fff9a895920) = -1 ENOTTY (Inappropriate ioctl for device) write(1, "Archive: dataset.zip\n", 22Archive: dataset.zip ) = 22 lseek(3, 943718400, SEEK_SET) = 943718400 read(3, "...


10

Linux doesn't have an exact equivalent of the DOS/Windows archive bit, but you can make something similar. Modern Linux systems support custom file attributes, at least on ext4 and btrfs. You can use getfattr to list them and setfattr to set them. Custom attributes are extended attributes in the user namespace, i.e. with a name of that starts with the five ...


9

You must create a level 0 backup first: $ tar --create --verbose --listed-incremental ./game.snar --gzip \ --file game_`date +%F`.tar.gz game/ and the next day, this command compress only files changed since the creation of the ./game.snar: $ tar --create --verbose --listed-incremental ./game.snar --gzip \ --file game_`date +%F`.tar.gz game/ This ...


9

This prints contents of ./x/y.txt from a.tar to STDOUT. tar xfO a.tar ./x/y.txt


8

tar is simply an archive format that is very good at exactly duplicating the files and preserving the directory tree and the original file attributes. TAR is very good for making backups, because everything is preserved. I use pbzip2 to compress the tar archives I use for system backups with very good results. this command should do the trick. tar -cpS "...


8

To repeat what others have said: we need to know more about the files that are being backed up. I'll go with some assumptions here. Append to the tar file If files are only being added to the directories (that is, no file is being deleted), make sure you are appending to the existing tar file rather than re-creating it every time. You can do this by ...


8

You can look for the cpio newc header (starting with 0707010): $ grep -abo 0707010 vmlinux.bin | head -n1 2531404:0707010 The -a (for all files even binary ones), -b (for byte offset), and -o (for only the matching part (and report the byte offset of the matching part instead of the line containing the matching part)) are non-standard GNU extensions to ...


8

In addition to all the other answers, I've recently struck a scripting situation where only one file was expected, but a previous employee wrote the scripts with the possibility of more than one file being generated. So files were tarred and bzipped, then transferred, and expanded. When the process grew to the point it made a 4.3 GB file, it rolled over ...


Only top voted, non community-wiki answers of a minimum length are eligible