Questions tagged [xz]

A container format for compressed streams. The .xz format can hold only a single file (it does not bundle multiple files into a single archive).

Filter by
Sorted by
Tagged with
0 votes
0 answers
44 views

how to extract a multi-threading for tar.xz

I used the following command to compress a drive mounted to a folder. based on How to use multi-threading for creating and extracting tar.xz tar -c -I 'xz -9 -T0' -f archive.tar.xz /var/lib/docker ...
user avatar
1 vote
0 answers
30 views

Reading from stdin `dd` copies more data than reading the same thing from file [duplicate]

I encountered weird behaviour of the dd tool: I downloaded compressed "newsystem.iso.xz". I ran xz -d --stdout newsystem.iso.xz | dd of=/dev/sdb bs=1M conv=sync. The decompressed "*....
user avatar
  • 21
1 vote
2 answers
214 views

Tar pipe to xz results in broken pipe

I have a backup script that runs the following commands: tar -c dir1 dir2 | xz -9 -T0 | gpg -c --batch --passphrase xxx | aws s3 ... The return values are always the same: tar fails with 141 (broken ...
user avatar
  • 2,301
1 vote
1 answer
287 views

7z command to create a XZ file

The man page says 7z tool supports 7z, LZMA2, XZ, ZIP, Zip64, CAB, RAR, ARJ, GZIP, BZIP2, TAR, CPIO, RPM, ISO, most filesystem images and DEB formats. And it says we could pass the type as -t{Type}. ...
user avatar
1 vote
1 answer
238 views

xz: OOM when compressing 1TB .tar

I'm trying to compress a large archive with multi-threading enabled, however, my system keeps freezing up and runs out of memory. OS: Manjaro 21.1.0 Pahvo Kernel: x86_64 Linux 5.13.1-3-MANJARO ...
user avatar
  • 113
1 vote
0 answers
41 views

Tar archive overwrote original file

I was compressing a rosbag file using tar when I mistakenly used the same original file name for the the archive name and had it overwritten by the output archive , the compressed archive size is very ...
user avatar
  • 11
5 votes
2 answers
984 views

How do I know whether tar really pass XZ_OPT to xz?

What I am doing: XZ_OPT='-T0 -9 -vv' tar -vvcJf ~/backup.tar.xz ...FILES I doubt if tar really pass the given options, I have tried those things: I used -vv in XZ_OPT, but there is no message from ...
user avatar
  • 1,059
0 votes
0 answers
501 views

Repairing a tar.xz file

I am trying to extract a xz archive, however, around half way through decompression I get the error Compressed data is corrupt. I have tried using xzcat and using the --ignore-check flag in case that ...
user avatar
  • 1
1 vote
1 answer
116 views

Installing m4 on OpenBSD

I know I have a lot more to learn, but I'm stuck at with something; how would I go about install m4 (https://directory.fsf.org/wiki/M4#tab=Overview) in OpenBSD? The thing is, from everything I've read,...
user avatar
2 votes
1 answer
619 views

How to check a tar.xz file to see if it's not corrupted

I have compressed some file with this command: XZ_OPT=-e9 tar cJf compressed_directory.tar.xz /path/to/directory Is there any way I could check these tar.xz files to see if they're not corrupted? I'm ...
user avatar
0 votes
1 answer
2k views

How do I get squashfs and XZ to work?

I'm trying to get snap on Linux Mint, and I've run into too many problems to count. The first thing I did was install the package (apt install snap), and then realised that it is blocked. So I ...
user avatar
2 votes
1 answer
943 views

Adjust logrotate to use xz for logs

On a current Debian Testing machine, I am wanting to use xz to compress log files instead of GZip. It has higher compression out of the box, and as such I wish to use it. However, whenever I specify ...
user avatar
  • 2,360
7 votes
1 answer
986 views

Is there a compression tool with an arbitrarily large dictionary?

I am looking for a compression tool with an arbitrarily large dictionary (and "block size"). Let me explain by way of examples. First let us create 32MB random data and then concatenate it to itself ...
user avatar
  • 323
0 votes
1 answer
249 views

How may I emulate the -R feature of "zip", but in gzip and xz?

The program "zip" has a -R feature which allows one to zip all files with a certain name in a directory tree: zip -r v/s zip -R For example: zip -R bigfile "bigfile" Will zip all of the following: ...
user avatar
3 votes
1 answer
8k views

Multithreaded xz, with gzip, pv, and pipes - is this the most efficient I can get?

I'm excited to learn that xz now supports multithreading: xz --threads=0 But now I want to utilise this as much as possible. For example, to recompress gzips as xz: gzip -d -k -c myfile.gz | pv | ...
user avatar
3 votes
1 answer
7k views

How do I `grep` through .xz compresses files?

I'm using Red Hat Virtualization (RHV) and it logs all of its files in this directory: $ pwd /var/log/vdsm $ ls | column -c 80 | head -10 backup vdsm.log.34.xz vdsm.log.69.xz import ...
user avatar
  • 346k
0 votes
0 answers
225 views

autocomplete with tar and .xz

When I wish to unpack a certain file with tar, I type tar followed by my chosen parameters. Then I type the first(s) letter of the file name and then press the tab key, as to let the autocomplete ...
user avatar
2 votes
1 answer
447 views

Docker busybox tar broken pipe

When executing docker run --rm -v ldaps_ldap-config:/volume -v /tmp/tmp.VznAcI3PTA:/backup busybox tar c -Jvf /backup/ldap-config.tar /volume I always get the following output: BusyBox v1.31.0 (...
user avatar
  • 403
0 votes
3 answers
604 views

Compress a large folder tree while keeping an easy way to navigate and extract particular files

I have a large and heavy folder tree with many files and folders. I want to compress it to reduce the storage size. This tree will not be frequently used any more, but quite probably I will need to ...
user avatar
  • 377
5 votes
1 answer
2k views

Writing to disk and compress with xz at the same time

I have a program that writes traces on disk and the size becomes very large. Normally, I use the following commands. ./run output.txt xz output.txt Can I pipe xz at the same time as output.txt is ...
user avatar
  • 1,015
1 vote
1 answer
853 views

GNU parallel proper usage in conjunction with tar, xz, gpg

I would like to speed up my archiving operations, I am usually doing 23 GiB (one Blu-Ray) backups. I have found this: How to do large file parallel encryption using GnuPG and GNU parallel? Since I ...
user avatar
0 votes
0 answers
544 views

How to make use of tar --verify directly?

This question is about compressing rather large data directories (around 22-23 GiB), specifically the integrity checks of such archives. I do these big backups about once a month, so the time it takes ...
user avatar
0 votes
2 answers
656 views

Unable to create tar with xz compression of a directory to another directory

I am trying to create a tar archive with xz compression of my home directory to another non-existent directory, but it gives me error: tar: Cowardly refusing to create an empty archive The command ...
user avatar
1 vote
4 answers
2k views

How to concatenate results of multiple commands and pipe into another without intermediate file?

Suppose I have four very large text files, all compressed with xz. file1.log.xz file2.log.xz file3.log.xz file4.log.xz What I'd like to do is concatenate the uncompressed contents of these four ...
user avatar
  • 2,408
2 votes
2 answers
622 views

Compressing Directory tree into multiple tar archives into a single xz

I'm trying to compress a large rootdir which contains many subdir_i the folder tree looks like: ./rootdir ./rootdir/subdir_1 ./rootdir/subdir_2 ... I am looking to output this to a single compressed ...
user avatar
  • 101
1 vote
0 answers
44 views

xzcmp does not work for some files. (Internal pipe error 141?)

I am using xzcmp to compare several XZ files. But I have a strange problem: For some files (*), xzcmp doesn't work and returns exit code 2, without error message. zxcmp version1.sql.xz version2.sql....
user avatar
5 votes
1 answer
4k views

Is there any way to check the compression ratio of a xz compressed file?

In windows by default one can see the compression ratio of zip files and after installing WinRAR it also supports this feature but currently I am using a linux and have compressed a file using lzma2 ...
user avatar
  • 145
7 votes
1 answer
3k views

Is there any way to verify the integrity of a xz archive without decompressing the entire archive?

Can I check the integrity of the archive without decompressing it as the archive is 64GB. It is a backup of my old laptop and before i format my old laptop I wanted to verify the backup but it will ...
user avatar
  • 145
7 votes
4 answers
5k views

How to convert all files from gzip to xz on the fly (and recursively)?

I have a directory tree with gzipped files like this: basedir/a/file.dat.gz basedir/b/file.dat.gz basedir/c/file.dat.gz etc. How can I convert all of these from gzip to xz with a single command and ...
user avatar
1 vote
3 answers
4k views

How to check what compression type an RPM package uses?

I recently realized that the RPM packages shifted from gzip-compression to xz-compression a few years ago. I need to check is the compression type of an RPM package that I have. I also need to check ...
user avatar
  • 511
1 vote
1 answer
830 views

'xz -d' vs. `unxz`

Why is xz -d preferred over unxz when writing scripts? From the man page for xz 5.1.0alpha: When writing scripts that need to decompress files, it is recommended to always use the name xz with ...
user avatar
1 vote
1 answer
2k views

compression levels gz and xz

Why isn't there any difference (output filesize) with these command lines? Gzip stdin The output files have same filesize even though the compression levels are different tar ... | gzip -c -1 > ...
user avatar
  • 1,677
8 votes
3 answers
23k views

RHEL6 + Error: xz compression not available

When installing packages via yum on a RHEL Server 6.6 system, I get the following error: $ sudo yum install foo Loaded plugins: product-id, rhnplugin, security, subscription-manager This system is ...
user avatar
4 votes
2 answers
6k views

XZ - Cannot allocate memory

If i run this command: XZ_OPT=-9 tar --xz -cvf files/compressed/xz/archive.tar.xz -C files/original/ . Get this message: xz: (stdin): Cannot allocate memory tar: files/compressed/lzma//archive.lzma:...
user avatar
0 votes
2 answers
701 views

Can I use lzma compression, without using "tar", on OSX 10.10.3?

I can create tar files compressed using LZMA as such: $ ls ./DbDumps alpha.dmp beta.dmp delta.dmp $ XZ_OPT=-9 tar cJf dbDumps.tar.xz DbDumps But, what I really want to do (but cannot) is compress ...
user avatar
  • 103
-1 votes
1 answer
4k views

Error: xz compression not available

I'm running redhat enterprise v6.6 64bit and am tyring to install 32bit Skype. In the process of trying to fix the dependency issue I managed to create another issue for myself. When I try and ...
user avatar
  • 1
232 votes
4 answers
83k views

Why are tar archive formats switching to xz compression to replace bzip2 and what about gzip?

More and more tar archives use the xz format based on LZMA2 for compression instead of the traditional bzip2(bz2) compression. In fact kernel.org made a late "Good-bye bzip2" announcement, 27th Dec. ...
user avatar
0 votes
1 answer
4k views

linux + xzcat command

Please advice from where ( which site ) I can download the xzcat Linux command? I am using linux version 5.4 I tried to download the xzcat from the site: http://www.filewatcher.com/m/xzcat.2-0....
user avatar
  • 43