Tag Info

Hot answers tagged

70

No. Unlike zip, gzip functions as a compression algorithm only. Because of various reasons some of which hearken back to the era of tape drives, Unix uses a program named tar to archive data, which can then be compressed with a compression program like gzip, bzip2, 7zip, etc. In order to "zip" a directory, the correct command would be tar -zcvf ...


45

XZ_OPT=-9 tar cJf tarfile.tar.xz directory tar's lowercase j switch uses bzip, uppercase J switch uses xz. The XZ_OPT environment variable lets you set xz options that cannot be passed via calling applications such as tar. This is now maximal See man xz for other options you can set.


32

You can pipe tar across an ssh session: $ tar czf - <files> | ssh user@host "cd /wherever; tar xvzf -"


32

Tar is an archiving tool (Tape ARchive), it only collects files and their metadata together and produces one file. If you want to compress that file later you can use gzip/bzip2/xz. For convenience, tar provides arguments to compress the archive automatically for you. Checkout the tar man page for more details.


31

Compression ratio is very dependent of what you're compressing. The reason text compresses down so well is because it doesn't even begin to fully utilize the full range of numbers representable in the same binary space. So formats that do (e.g compressed files) can store the same information in less space just by virtue of using all those binary numbers that ...


29

Globs are not regular expressions. In general, the shell will try to interpret anything you type on the command line that you don't quote as a glob. Shells are not required to support regular expressions at all (although in reality many of the fancier more modern ones do, e.g. the =~ regex match operator in the bash [[ construct). The .??* is a glob. ...


26

There are several different patterns for options that have been used historically in UNIX applications. Several old ones, like tar, use a positional scheme: command options arguments as for example tar uses tar *something*f "file operated on" *"paths of files to manipulate"* In a first attempt to avoid the confusion, tar and a few other programs ...


24

This should work: mkdir pretty_name && tar xf ugly_name.tar -C pretty_name --strip-components 1 -C changes to the specified directory before unpacking (or packing). --strip-components removes the specified number of directories from the filenames stored in the archive. Note that this is not really portable. GNU tar and at least some of the BSD ...


23

tar stores relative paths by default. GNU tar even says so if you try to store an absolute path: tar -cf foo.tar /home/foo tar: Removing leading `/' from member names If you need to extract a particular folder, have a look at what's in the tar file: tar -tvf foo.tar And note the exact filename. In the case of my foo.tar file, I could extract ...


22

That's actually a feature, not a problem. Archives with absolute locations are a security risk. Attackers could use such archives to trick users into installing files in critical system locations. Yes, you could use -P. But what's wrong with allowing tar to remove the forward slash, and simply requiring the user of the archive to explicitly do the ...


20

I think you are looking for pbzip2: PBZIP2 is a parallel implementation of the bzip2 block-sorting file compressor that uses pthreads and achieves near-linear speedup on SMP machines. Have a look at the project homepage or check your favorite package repository.


18

It's a common convention to use - in a filename argument to mean stdin or stdout; tar follows the same convention. From the man page: -f, --file [HOSTNAME:]F use archive file or device F (default "-", meaning stdin/stdout)


18

tar is one of those ancient commands from the days when option syntax hadn't been standardized. Because all useful invocations of tar require specifying an operation before providing any file name, most tar implementations interpret their first argument as an option even if it doesn't begin with a -. Most current implementations accept a -; the only ...


18

Assuming xz honors the standard set of commandline flags - including compression level flags, you could try: tar -cf - foo/ | xz -9 -c - > foo.tar.xz


17

To tar and gzip a folder, the syntax is: tar czf name_of_archive_file.tar.gz name_of_directory_to_tar The - is optional. If you want to tar the current directory, use . to designate that. To construct your filename, use the date utility (look at its man page for the available format options). For example: cd /var/www && sudo tar czf ...


16

You posted in a comment that you are working on a Mac OS X system. This is an important clue to the purpose of these ._* files. These ._* archive entries are chunks of AppleDouble data that contain the extra information associated with the corresponding file (the one without the ._ prefix). They are generated by the Mac OS X–specific copyfile(3) family of ...


16

tar has an option to suppress this message [1]: -m, --touch don't extract file modified time However, you should probably also check that you don't have an issue with your system clock. [1] http://unixhelp.ed.ac.uk/CGI/man-cgi?tar


16

tar -c data_dir | wc -c without compression or tar -cz data_dir | wc -c with gzip compression or tar -cj data_dir | wc -c with bzip2 compression will print the size of the archive that would be created in bytes, without writing to disk. You can then compare that to the amount of free space on your target device. You can check the size of the data ...


15

There are many ways to do what you want. The simplest is to use a pìpe: tar zcvf - MyBackups | ssh user@server "cat > /path/to/backup/foo.tgz" Here, the compression is being handled by tar which calls gzip (z flag). You can also use compress (Z) and bzip (j). For 7z, do this: tar cf - MyBackups | 7za a -si -mx=9 -ms=on MyBackups.tar.7z | ssh ...


14

From man tar: -C directory In c and r mode, this changes the directory before adding the following files. In x mode, change directories after opening the archive but before extracting entries from the archive. i.e, tar xC /foo/bar -f /tmp/foo.tar.gz should do the job. (on FreeBSD, but GNU tar is basically the same in this ...


14

Simple example, creating a gzipped tarball out of a directory, excluding a subdirectory and listing the contents: $ ls fruit/ apple banana peach tomato $ tar czf onlyfruit.tar.gz --exclude=tomato fruit/ $ tar tf onlyfruit.tar.gz fruit/ fruit/peach/ fruit/apple/ fruit/banana/


14

When you send the same set of files, rsync is better suited because it will only send differences. tar will always send everything and this is a waste of resources when a lot of the data are already there. The tar + rsync + untar loses this advantage in this case, as well as the advantage of keeping the folders in-sync with rsync --delete. If you copy the ...


14

If you want to exclude an entire directory, your pattern should match that directory, not files within it. Use --exclude=/data/sub1 instead of --exclude='/data/sub1/*' Be careful with quoting the patterns to protect them from shell expansion. See this example, with trouble in the final invocation: $ for i in 0 1 2; do mkdir -p /tmp/data/sub$i; echo foo ...


14

The -f option should directly precede the filename. So, use tar -vczf filename.tar.gz instead of -vcfz


12

7zip can run on multiple threads when given the -mmt flag, but only when compressing into 7z-archives which offer great compression but are generally slower to create then zip archives. Do something like this. 7z a -mmt foo.7z /opt/myhugefile.dat


12

Tar with bzip2 compression should take as much load off the network and on the cpu. $ tar -C /path/to/src/dir -jcf - ./ | ssh user@server 'tar -C /path/to/dest/dir -jxf -' Not using -v because screen output might slow down the process. But if you want a verbose output use it on the local side of tar (-jcvf), not on the remote part. If you repeatedly copy ...


12

With GNU tar ≥1.16, use --transform to apply a sed regexp transformation to each file name (the transformation is applied on the full path in the archive): tar xf foo.tar --transform 's!^ugly_name\($\|/\)!pretty_name\1!' If ugly_name is the toplevel component of all the file names in the archive, you don't need to bother with precise matching: tar xf ...


12

I've just had exactly this problem. As Gilles suggested, upgrading tar is the answer but (surprise surprise) tar can't be upgraded in the usual way because dpkg requires version 1.23 or later before it'll unpack and install the latest tar deb. dpkg really needs an explicit dependency to ensure that when a later version of dpkg is installed, the latest tar ...


12

If you want to get rid of "Removing leading `/' from member names" being printed to STDERR, but still want to leave off those leading slashes as tar wisely does by default, I saw an excellent solution here by commenter timsoft. The solution involves using -C option to change directory to the root (/), then specifying the file tree to archive without a ...


12

You compression attempts failed because your data is already highly compressed and there's not much more to gain, see the other answers for more detailed explanations. However, if you can agree on lossy compression, in contrast to lossless like you tried before, you can compress the images significantly. But since data is cut away, it can not be undone. ...



Only top voted, non community-wiki answers of a minimum length are eligible