Can losetup be made efficient with sparse files?

Question

So my setup is like this.

$ truncate -s 1T volume
$ losetup -f --show volume
/dev/loop0
$ mkfs.ext4 /dev/loop0
$ ls -sh volume
1.1G volume
$ mount /dev/loop0 /mnt/loop

Now I have a 1.1TB volume, as expected. The overhead of ext4 expanded the sparse file to 1.1G, but that's fine. Now to add a file.

$ dd if=/dev/urandom of=/mnt/loop/file bs=1M count=10240
$ ls -sh volume
12G volume

Cool, now I don't want the file.

$ rm /mnt/loop/file
$ ls -sh volume
12G volume

The free space is still taking up space, as expected, and $ fallocate -d volume frees up 1gb.

My question is, how can I zero out the free space here without expanding the volume to the full size? $ dd if=/dev/zero will expand it to full size, and with conv=sparse makes it create a useless sparse file inside the volume.

TL;DR: Is there a way to make losetup ignore writes of null blocks to null sectors, while allowing everything else?

Answer 1

To automatically discard data blocks when they are no longer used, use mount -o discard .... Or you can manually run fstrim.

This feature was apparently added to the loop device in Linux 3.2. https://outflux.net/blog/archives/2012/02/15/discard-hole-punching-and-trim/

---

In the general case, mount -o discard is not guaranteed to be effective, because some types of device are allowed to ignore discard requests when they are busy. This would not be a concern for the size of your sparse file though.

In this general case - e.g. if you also wanted to send discard requests to an underlying physical device - the most robust method is to run fstrim at regular intervals.

Answer 2

Not a solution using built in tools, but I whipped up a python script to do what I want. Here it is if it helps anyone. Still looking for a built in solution, if there is one.

#!/usr/bin/python3.6
import os
import time
import sys

if len(sys.argv) != 2:
    print('''Usage: {} <file>

Fills a file/device with zeros, with efficiencies for sparse files.'''.format(sys.argv[0]))
    exit(1)

f = open(sys.argv[1], 'rb+', 0)

bs = 1024*1024 # block size
zb = b'\0'*bs  # a full block of zeros

lasttime = round(time.time())
block = f.read(bs)
while len(block) == bs:

    # IFF a block contains non-zero characters, zero it
    if block != zb:
        f.seek(-bs, os.SEEK_CUR)
        f.write(zb)
    block = f.read(bs)

    # Print the processed bytes every second
    # Could add a MB/s or % completed, but too lazy
    if round(time.time()) != lasttime:
        print(f.tell())
        lasttime = round(time.time())

# Hit EOF, might have missed bytes at the end
# Zero last block unconditionally, too lazy to check if its required
f.seek(-bs, os.SEEK_CUR)
f.write(zb)
f.close()