I have an aging Centos NFS server with a few different ZFS datasets. I've been migrating the data to a newer NFS server using rsync. The new server is also a Centos ZFS host. My issue is that for one dataset, the referenced size is drastically different between old and new servers, and I can't determine the reason.
I'm not using compression or deduplication. I expected the zfs properties to be the same but I noticed the old host does have the following properties enabled for the dataset (devices, exec and setuid). The only other difference I found is the CentOS and ZFS version:
Old host: zfs 0.6.5, centos 7.3, zfs list shows 2.00T referenced
New host: zfs 0.7.2 centos 7.4, zfs list shows 1.29T referenced
To copy the data, I mounted the old export onto the new server and used the following rsync options: -avhH --delete. Looking at df, the inode count is the same but df does show a big difference in used space (1.3T vs 2T). I used 'find' to build a csv of all files in the dataset along with their size and disk usage. The row count and sum of the size column is identical between the two hosts (the size sum is about 1.29TB) The disk usage is only slightly different between the two hosts and matches closely with 1.3T. I've also tried running lsof on the old server to see if there is some unlinked file that was consuming the difference in capacity but nothing was returned as '(deleted)'... any other ideas or suggestions that might explain the difference?