2

I have a 16M file.

I take a snapshot of the ZFS filesystem which contains it.

If I overwrite the file with the same data, will ZFS need to store two copies of all of the blocks of the file?

3

Yes, ZFS will allocate extra space for the same file if a snapshot is taken from the fileset.

First let's create an empty fileset and prepare a file to copy from. For simplicity reasons, compression was turned off and the pool was created on a single disk without any raidz or mirror.

[root@localhost ~]# dd if=/dev/urandom of=/tmp/testfile bs=16M count=1
1+0 records in
1+0 records out
16777216 bytes (17 MB) copied, 0.113345 s, 148 MB/s    


[root@localhost ~]# zpool create tank sdd
[root@localhost ~]# zfs create tank/test

The space used on disk can be seen with zpool list.

[root@localhost]# zpool list tank
NAME     SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
tank    9.94G   182K  9.94G         -     0%     0%  1.00x  ONLINE  -

Now copy the file over to the ZFS fileset, create a snapshot and look at the space used.

[root@localhost ~]# /bin/cp /tmp/testfile /tank/test/

[root@localhost ~]# zfs list -t all -r tank/test
NAME        USED  AVAIL  REFER  MOUNTPOINT
tank/test  16.0M  9.61G  16.0M  /tank/test

[root@localhost ~]# zfs snapshot tank/test@1
[root@localhost ~]# zfs list -t all -r tank/test
NAME          USED  AVAIL  REFER  MOUNTPOINT
tank/test    16.0M  9.61G  16.0M  /tank/test
tank/test@1     0B      -  16.0M  -

Okay, copy the same file again to the same location in the ZFS fileset and look at the space used again.

[root@localhost ~]# /bin/cp -f /tmp/testfile /tank/test/
[root@localhost ~]# zfs list -t all -r tank/test
NAME          USED  AVAIL  REFER  MOUNTPOINT
tank/test    32.0M  9.60G  16.0M  /tank/test
tank/test@1  16.0M      -  16.0M  -

Also the disk space used in the pool is growing to 32MB.

[root@localhost tank]# zpool list tank
NAME     SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
tank    9.94G  32.2M  9.91G         -     0%     0%  1.00x  ONLINE  -

As you can see, the tank/test fileset now accounts for 32MB in total, splitting up in 16MB in the tank/test fileset and 16MB in the tank/test@1 snapshot. Also the output of zpool list shows an allocation of 32MB on disk.
If you repeat copy and take snapshot the tank/test total USED would grow further.

Update

Thanks @Andrew Henle for asking. Had to update my answer above and will continue the deduplication stuff below.

Let's do this again with a fileset that has deduplication enabled.

[root@localhost]# zfs destroy -r tank/test
[root@localhost]# zfs create tank/test-dedup
[root@localhost]# zfs set dedup=on tank/test-dedup

I will skip the single steps here and will just add the output of the overview of the used space.

[root@localhost ~]# zfs list -t all -r tank/test-dedup
NAME                  USED  AVAIL  REFER  MOUNTPOINT
tank                 32.3M  9.61G    24K  /tank
tank/test-dedup      32.1M  9.61G  16.0M  /tank/test-dedup
tank/test-dedup@1    16.0M      -  16.0M  -
[root@localhost ~]# zpool list tank
NAME     SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
tank    9.94G  16.3M  9.92G         -     0%     0%  2.00x  ONLINE  -

With deduplication enabled, the snapshot is still reported to use the extra size, but as you can see with zpool list the deduplication is saving the space on disk and gives a deduplication ratio of 2.

IIRC in general it is not recommended to use deduplication due to heavy memory usage and the resulting performance impact. I think there is work going on to improve the deduplication feature in ZFS.

  • Is that with or without deduplication enabled? – Andrew Henle Mar 2 '18 at 13:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.