Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

So I need to compress a directory with max compression.

How can I do it with xz? I mean I will need tar too because I can't compress a directory with only xz. Is there a oneliner to produce e.g. foo.tar.xz?

share|improve this question
    
FWIW, man 1 xz says it's not a good idea to blindly use -9 for everything like it often is with gzip(1) and bzip2(1). -7 ... -9 [...] These are useful only when compressing files bigger than 8 MiB, 16 MiB, and 32 MiB, respectively. RTFM for more info. –  cychoi Feb 10 at 7:42

5 Answers 5

up vote 19 down vote accepted

Assuming xz honors the standard set of commandline flags - including compression level flags, you could try:

tar -cf - foo/ | xz -9 -c - > foo.tar.xz 
share|improve this answer
    
and this uses maximum compression level with XZ? –  LanceBaynes Jan 12 '12 at 21:34
    
adding -9 to xz will make it max –  bdowning Jan 12 '12 at 21:45

With a recent GNU tar on bash or derived shell:

XZ_OPT=-9 tar cJf tarfile.tar.xz directory

tar's lowercase j switch uses bzip, uppercase J switch uses xz.

The XZ_OPT environment variable lets you set xz options that cannot be passed via calling applications such as tar.

This is now maximal.

See man xz for other options you can set (-e/--extreme might give you some additional compression benefit for some datasets).

share|improve this answer
    
and this uses maximum compression level with XZ? –  LanceBaynes Jan 12 '12 at 21:34
    
It does now, see edited answer and XZ_OPT env var ;) –  bdowning Jan 13 '12 at 0:08
8  
No, you don't. That's the whole point. You can set the environment var for just that invocation. You can export it if you want to, but you don't have to. –  bdowning Apr 23 '13 at 9:36
1  
You're assuming bash-like shell for that. –  anddam Apr 29 '13 at 19:56
1  
@anddam, that's supported by all shells of the Bourne family (Bourne, ksh, mksh, pdksh, ash, dash, bash, yash, zsh) and rc and akanga. fish, csh, tcsh and es being the major shells that don't support it. There, you'd use the env command. –  Stéphane Chazelas Jan 20 at 10:33

You might try different options, for me -4e works better

tar cf - wam_GG_${dir}.nc | xz -4e > wam_GG_${dir}.nc.tar.xz 

I tested by running:

$ tar -cf - wam_GG.nc | xz -4e > wam_GG.nc.xz
$ tar -cf - wam_GG.nc | xz -9e > wam_GG.nc.xz.2

So, it seems that option -4e works a little bit better than -9e.

$ ll wam_GG.nc.xz*
-rw-rw-r--. 1 504 504 2707596 Jan 16  2015 wam_GG.nc.xz
-rw-rw-r--. 1 504 504 2708416 Jan 16  2015 wam_GG.nc.xz.2
share|improve this answer
1  
This really doesn't answer the question. This is just an observation that for your particular small data set, -4e already gets the best compression and so the higher levels don't get any more benefit ( and even an ever so slight penalty ). –  psusi Jan 16 at 16:00
    
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post - you can always comment on your own posts, and once you have sufficient reputation you will be able to comment on any post. –  HalosGhost Jan 16 at 16:33
    
Are you the same user as Szymon Roziewski? If so, please don't post multiple answers. Instead, edit your original answer. If you can't access your first account, please see here for how to merge your accounts. In the meantime, I am deleting your previous answer and including it here. –  terdon Jan 16 at 16:35
    
Ok, I have done a more comprehensive study on that. What I got is here. I chose some files from my hardrive and made compression with option -4e and -9e. So, it's better to find your best solution by yourself. You were right, for some cases -9e is better whereas for another it's not: no difference = 660 4e better than 9e = 74 9e better than 4e = 17 total files = 751 tar 2 html 2 csv 2 xml 2 gz 2 ppt 2 eps 2 docx 2 gif 2 rpm 3 png 3 asv 3 xlsx 3 exe 3 rar 4 nc 4 txt 5 odt 6 xls 7 zip 7 doc 9 m 12 dat 17 other 109 pdf 133 135 jpg 270 –  Szymon Roziewski Jan 20 at 9:51
    
(comments may be edited only for 5 minutes) txt 109 txt/pdf 135 –  Szymon Roziewski Jan 20 at 9:59
XZ_OPT=-9e tar cJf tarfile.tar.xz directory

is even better than

XZ_OPT=-9 tar cJf tarfile.tar.xz directory
share|improve this answer

tar command uses J flag for xz files. An example:

tar -cJvf foo.tar.xz foo/

share|improve this answer
    
The J was already mentioned in bdowning's answer –  Anthon Jan 8 '14 at 22:58

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.