4

I installed on Debian Wheezy zfs-fuse file system and enabled compression gzip-9 on one dataset ("storage/backup"). When I check if the compression is enabled on this dataset, it shows YES:

$: zfs get compression storage/backup
NAME         PROPERTY     VALUE     SOURCE
storage/backup  compression  gzip-9    local

However, when I check the compression rate with du -ah or with sfx get compressratio no any compression can be seen.

All files, including well compressible ones (e.g. text files), take up exactly the same disk size as uncompressed ones:

$: zfs get compressratio storage/backup
NAME         PROPERTY       VALUE  SOURCE
stor/backup  compressratio  1.00x  -

Why does this situation occur?

Here some info from zfs get all about the dataset:

compressratio         1.00x                  -
mounted               yes                    -
quota                 none                   default
reservation           none                   default
recordsize            128K                   default
mountpoint            /storage/backup        default
sharenfs              off                    default
checksum              on                     default
compression           gzip-9                 local
atime                 on                     default
devices               on                     default
  • 2
    ZFS only compress new files only – Arch user Jul 3 '14 at 8:39
  • All the files are new on that dataset. There are copied ones and newly created ones - neither ones are compressed. – Al Berger Jul 3 '14 at 8:52
  • Are the files already compressed? – Matt Jul 3 '14 at 8:55
  • @mtm No. E.g. I created a new file, entered 8000 instances of one character - 'du -ah' shows that this file has 8 Kb size. – Al Berger Jul 3 '14 at 9:00
  • Also, a newly created file containing 1 000 000 of one character has 980 kb storage size in 'du -ah'. – Al Berger Jul 3 '14 at 9:21
1

It looks like zfs-fuse will update the compressratio data every 30 seconds with limited IO occurring but there is another trigger to the update as background IO or really large files cause the data update to occur sooner.

I've put some test functions up on a gist. They require a clean (no files) file system that will start at 1.00x.

If the scripts pause forever on the first test then your compression counters are never updating and you have an issue with your install.

Running the scripts on a Debian wheezy box:

$ uname -a
Linux zfs-fuse 3.2.0-4-686-pae #1 SMP Debian 3.2.54-2 i686 GNU/Linux

Results in the following:

$ test_compression compress
Testing [compress]

Testing size [4096]
Waited 0 seconds for [compressratio_is_one]
4096 bytes made up of 1*4096 blocks
Waited 20 seconds for [compresstario_is_not_one]
1.12x

Testing size [16384]
Waited 30 seconds for [compressratio_is_one]
16384 bytes made up of 1*16384 blocks
Waited 30 seconds for [compresstario_is_not_one]
1.53x

Testing size [1048576]
Waited 30 seconds for [compressratio_is_one]
1048576 bytes made up of 1*131072 blocks
Waited 30 seconds for [compresstario_is_not_one]
31.44x

Testing size [33161216]
Waited 30 seconds for [compressratio_is_one]
33161216 bytes made up of 255*131072 blocks
Waited 0 seconds for [compresstario_is_not_one]
202.31x

You can reduce this, normally by about half by doing something intensive in the background which probably triggers the counter update.

In the background

$ while true; do touch somefile; rm somefile; done

Then testing again:

$ test_compression compress
Testing [compress]

Testing size [4096]
Waited 0 seconds for [compressratio_is_one]
4096 bytes made up of 1*4096 blocks
Waited 5 seconds for [compresstario_is_not_one]
1.11x

Testing size [16384]
Waited 17 seconds for [compressratio_is_one]
16384 bytes made up of 1*16384 blocks
Waited 17 seconds for [compresstario_is_not_one]
1.50x

Testing size [1048576]
Waited 16 seconds for [compressratio_is_one]
1048576 bytes made up of 1*131072 blocks
Waited 10 seconds for [compresstario_is_not_one]
29.73x

Testing size [33161216]
Waited 0 seconds for [compressratio_is_one]
33161216 bytes made up of 244*131072 blocks
Waited 0 seconds for [compresstario_is_not_one]
201.35x

Of note, on FreeBSD the update happens ~ every 5 seconds:

$ test_compression giggidy/compress
Testing [giggidy/compress]

Testing size [4096]
Waited 0 seconds for [compressratio_is_one]
4096 bytes made up of 1*4096 blocks
Waited 4 seconds for [compresstario_is_not_one]
1.21x

Testing size [16384]
Waited 5 seconds for [compressratio_is_one]
16384 bytes made up of 1*16384 blocks
Waited 5 seconds for [compresstario_is_not_one]
1.91x

Testing size [1048576]
Waited 5 seconds for [compressratio_is_one]
1048576 bytes made up of 1*131072 blocks
Waited 5 seconds for [compresstario_is_not_one]
39.33x

Testing size [33161216]
Waited 5 seconds for [compressratio_is_one]
33161216 bytes made up of 1*131072 blocks
Waited 4 seconds for [compresstario_is_not_one]
114.25x

I will add a Solaris based example when I can get on a box.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.