New answers tagged

-2

If you can, I would gzip the files on the NAS directly one by one, then tar them together. Essentially, your main problem seems to be taring and compressing at the same time. Taring is pretty fast on it's own, but when you tar and compress, you make a 200G file then compress that. Do it the other way. Make 150 small files then tar them together. If you ...


1

Unless it is a vendor provided package that integrates with the OS package manager you shouldn't extract it in that location. Please go to /opt and extract it there for example and set up your environments variables as shown on the download page for Swift. By doing a full extraction in /opt, which is there for third-party applications, you can also quickly ...


0

When extracting files, if tar discovers that the extracted file already exists, it normally replaces the file by removing it before extracting it, to prevent confusion in the presence of hard or symbolic links. (If the existing file is a symbolic link, it is removed, not followed.) However, if a directory cannot be removed because it is nonempty, tar ...


1

If the only requirement is that the compression is fast, I'd recommend lz4 very highly. It's used in lots of places where the speed of compression is more important than the compression ratio (e.g. filesystems with transparent compression like ZFS)


20

The first step is to figure out what the bottleneck is: is it disk I/O, network I/O, or CPU? If the bottleneck is the disk I/O, there isn't much you can do. Make sure that the disks don't serve many parallel requests as that can only decrease performance. If the bottleneck is the network I/O, run the compression process on the machine where the files are ...


2

You can reduce the amount of compression (in terms of space saved) to make it faster. To start with, bzip2 is MUCH slower than gzip, though it compresses smaller. You can also change the compression level of bzip2, gzip, or most compression programs to trade size for speed. If you are not willing to trade size of speed, you can still probably get the same ...


10

You can install pigz, parallel gzip, and use tar with the multi-threaded compression. Like: tar -I pigz -cf file.tar.gz * Where the -I option is: -I, --use-compress-program PROG filter through PROG Of course, if your NAS doesn't have multiple cores / powerful CPU, you are limited anyway by the CPU power. The speed of the hard-disk/array on which the ...


2

You are using printf to escape the spaces, as well as quoting it "" If you omit the printf call, and use the original $d with double-quotes, you are good. f=`echo $d | tr ' ' '_' | tr -d ',.!'` # Clean the name t=$YEAR_DIR/$f.tar.gz ##d=$(printf '%q' "$d") escapes the spaces # echo tar czf $t "$d" # This outputs the command as expected ...


2

No, in general this isn't possible. A FTP server usually has commands to get information about files and directories and to store, retrieve, delete and rename files. Commands to mount devices and to send messages to users are also standardized but not implemented in current servers. See the list of FTP commands on Wikipedia for details. No RFC mentions a ...


-3

This is a well known pitfall in the definition of the tar command line syntax. The problem can be avoided by using my star instead. star uses a different much less error sensible CLI definition when you call it under the name star and it is still much safer than the traditional tar implementation as it will not overwrite plain files when called as tar.


0

Explanation: The f in cvf is a shortcut for the -f option, which expects the name of the target file to tar to as next parameter. The outcome of switching the order of parameters has already been described by @AndrewHenle and @StephenKitt.


4

You asked tar to archive the files file2 and total.tar in the archive called file1, which it attempted to do. Unfortunately that means that file1 was overwritten, all you can get from it now is file2: tar tvf file1 (don't add a z in there, you didn't specify it when creating the archive). The only way you'll recover file1 is from backups.


7

In addition to all the other answers, I've recently struck a scripting situation where only one file was expected, but a previous employee wrote the scripts with the possibility of more than one file being generated. So files were tarred and bzipped, then transferred, and expanded. When the process grew to the point it made a 4.3 GB file, it rolled over ...


2

I would tar a single file, to copy it preserving the timestamp (which is easily overlooked in downloads). File permissions and ownership are less important: download is a term that applies to systems which are not well integrated. Whether tar'd or not, it is standard practice to compress the file to make downloads faster — and avoid running out of ...


-1

Tar is especially useful for multiple files not written to a formal file system, it always has been. If for some reason there is on occasion, only 1 file to be written it is of no real consequence. I can dd my .tar.gz directly to /dev/sdx without regard to partition or file system. It may as well be tape. It is generally done because the script or process ...


11

What I was missing was --group=root in addition to --owner=root. tar -c --{owner,group}=root (possibly with an optional --numeric-owner) fully anonymizes the archive.


60

You are actually asking only half of the question. The other question being, "Why would I compress a tar file with gzip?". And the answer is not just that gzip makes the file smaller (in most cases): tar: stores filename and other metadata: mode, owner ID, group ID, filesize, modification time stores a checksum (for the header only) gzip: can store ...


6

You can use --numeric-owner, that will just put your UID (1000 or something similar on most systems) in the file. From man tar: --numeric-owner always use numbers for user/group names


17

There is an important difference that could make using tar important under some circumstances: Besides the "metadata" that @jofel mentioned in his answer, tar records the filename in the archive. When you extract it, you get the original filename regardless of what the archive is called. In your case the tar archive and the file it contains have the related ...


29

There is a quite big advantage to using only-gzipped text files - the contents can be directly accessed with command-line tools like less, zgrep, zcat.


21

I would say it's likely that the people just don't realise they can use gzip/bzip2/xz without tar. Possibly because they come from a DOS/Windows background where it is normal for compression and archiving to be integrated in a single format (ZIP, RAR, etc). While there may be slight advantages to using tar in some situations due to the storage of metadata ...


158

Advantages of using .tar.gz instead of .gz are that tar stores more meta-data (UNIX permissions etc.) than gzip. the setup can more easily be expanded to store multiple files .tar.gz files are very common, only-gzipped files may puzzle some users. (cf. MelBurslans comment) The overhead of using tar is also very small. If not really needed, I still do ...


6

To cover a few points the other answers haven't: First, look what's in the file before you extract it: tar -tvf untrusted_tar_file.tar If there's anything in there you don't trust or want to extract, don't extract the tarball. Second, extract the tarball as a non-root user that only has write access to the one directory you're extracting the tarball ...


18

With GNU tar, it's simply tar -xvf untrusted_file.tar in an empty directory. GNU tar automatically strips a leading / member names when extracting, unless explicitly not told otherwise with the --absolute-names option. GNU tar also detects when the use of ../ would cause a file to be extracted outside of the toplevel directory and puts those files in the ...


34

You don't need the paranoia at all. GNU tar — and in fact any well-written tar program produced in the past 30 years or so — will refuse to extract files in the tarball that begin with a slash or that contain .. elements, by default. You have to go out of your way to force modern tar programs to extract such potentially-malicious tarballs: both GNU and BSD ...


-1

Given that you get the error Not a directory, it is obvious that the symlink in the tar archive is lib/jre/ -> lib/jre1.8.0_46 which is different from what you tell us in your text. If gtar does not complain about that link, it is broken. Unfortunately, you did not give sufficient information about your constraints. We need to know: what type of ...


3

tar: option requires an argument -- 'f' This gives it away -- tar's 'f' flag specifies the file to read or create. Since you were piping a (compressed) tar file in from curl, you just needed to tell tar that the "file" to read was stdin with a '-'.


2

The root cause of this problem is amazingly short: . (yes: a dot). Understand that find (without a dir) is equivalent to find .. From man find: If no paths are given, the current directory is used. And, when you execute find . the dot appears in the generated list ( Using only four files with distinct names to make it simple ): $ find ...


4

Your problem is that ls -lR will be executed for all files (which will display the files) and every directory (which will display the contents of the directory). If your directory-hierarchy would not be flat, but contain sub-directories, this would display the contents even more often, as -R tells ls to traverse subdirectories again. Instead you should ...


4

The problem is that find finds the Webcam directory, too, and runs ls Webcam which lists all the files there. To only list files, not directories, tell find -type f


0

Another solution that works is zgrep zgrep -r filename *.zip


2

You need to extract them, but you don't need to store them on the disk : tar -xOf MainFile.tgz SubFile1.tgz | tar -xO SubFile2.tgz | tar -xO SubFile3.tgz | tar -x abc.txt The -O flags sets output to stdout and without -f tar will accept archive data from stdin.


-1

Whats is the difference? How can I possibly open the .b tar? As far as I can tell, either ClockworkMod or TAR splits the backup into several files. If you're able to open the data.ext4.tar.a file, it might be because it has the TAR file headers at its start while data.ext4.tar.b does not. To get to the data in data.ext4.tar.b, you probably have to ...


2

It sounds like something split the file. When you try to un-tar the .a file does it abort early? That's a hint that it is split. Is the .a file 4GB? That's another hint. Try concatenating the files on a machine that can handle large files... cp myfile.tar.a myfile_full.tar cat myfile.tar.b >> myfile_full.tar tar xvf myfile_full.tar


1

According to How do you extract an App's data from a full backup made through “adb backup”?, that is a compressed tar-file using the deflate method. The accepted answer in that thread points to a program which you might use: nelenkov/android-backup-extractor There also is this, which might work: Android Backup Extractor. The deflate method is one of those ...


1

I have done a lot of research on this. You can do a test on the file with a word count but it will not give you the same number number as a du -sb adir. tar -tvOf afile.tar | wc -c du counts every directory as 4096 bytes, and tar counts directories as 0 bytes. You have to add 4096 to each directory: $(( $(tar -tvOf afile.tar 2>&1 | grep '^d' | wc ...


3

Depending on the source (and weight you may attach), shar dates back to around 1980. In a form which you might recognize, that comes from Rich Salz's implementation introduced in 1988, and improved in stages over the next few years. shar was originally a convenience for bundling text files. uuencoding (a way to send binary files) has been around at least ...



Top 50 recent answers are included