Is there a command line tool that can be used to rewrite all regular files in a directory tree either in-place or by creating new inodes?
With rewriting a file in-place, I mean opening the file for reading and writing, reading blocks of a reasonable size and writing those blocks at the same location, doing this for the whole file. Basically what this command line does:
find dir -type f -print0 | xargs -0 -n1 bash -c 'dd if="$1" of="$1" conv=notrunc bs=64M' -
If instead a new inode is created, file attributes should be replaced as good as possible, e.g. what this command does:
find dir -type f -print0 | xargs -0 -n1 bash -c 'echo "$1"; cp -a "$1" "$1~" && mv "$1~" "$1"' -
I'm in the process of trying to gain some experience and finding good practices for using ZFS deduplication, where appropriate. ZFS deduplication uses a DDT (deduplication table) and operates on blocks of a size given by the file's recordsize, which has an impact on the effectiveness and memory-usage of deduplication. I'm exploring the possibilities of migrating already-written data to use or stop using the DDT or change the file's recordsize. ZFS does not automatically change these parameters of already-written data, so the data needs to be rewritten.
To change whether the DDT is used, it is sufficient to rewrite the data in place (without creating a new file). But the recordsize of a file is determined when it is created and thus a new file needs to be created to change it.