4

I have a filesystem handled by ZFS and it's about 8 GB.

I tried to zero out the filesystem by running:

dd if=/dev/zero of=/EMPTY bs=1M | true
rm -f /EMPTY

And normally on an ext4 filesystem, it completes in a few seconds.

But this time it just kept going.

I was able to launch a second terminal and see what the size of /EMPTY was. And after running ll -h, it said that /EMPTY was 800 GB! Obviously 800 GB does not fit into 8 GB.

Furthermore by running df -h, it showed that the filesystem wasn't getting filled up.

However the dd as consuming 50% of the CPU.. etc, heating up my laptop. I'm assuming it was doing a lot of work but achieving nothing.

So why is dd running forever creating a file that doesn't take up any size? On ext4 it's meant to complete and give me an error so I can proceed.

6
  • Can you run strace or truss and see what is happening behind the scenes?
    – Warwick
    Commented Jul 7, 2014 at 6:08
  • Sounds like either ZFS is doing transparent compression (the same character, NULL, over and over compresses extremely well), or ZFS is smart and is creating a sparse file.
    – phemmer
    Commented Jul 7, 2014 at 6:21
  • That's probably it. I have compression switched on, and I read something about sparse files. Then the question, how to resolve and prevent this. I'm now trying on a different directory like ~/EMPTY instead of /EMPTY. Commented Jul 7, 2014 at 6:34
  • @Patrick ok, no matter what df-h doesn't show an increase in size. So I guess zeroing out is impossible with ZFS when there's compression on? Commented Jul 7, 2014 at 7:22
  • Do you want to create a big file on top of the filesystem, or do you want to wipe the whole filesystem itself?
    – LatinSuD
    Commented Jul 7, 2014 at 15:48

3 Answers 3

6

I don't know what you are trying to achieve so I can only describe what your command is actually doing.

dd if=/dev/zero

You are reading from a special device which returns an infinite number of zero (or NUL) bytes.

of=/EMPTY

You are creating a new file using the above (infinite) input.

bs=1M

You read and write the infinite data stream in megabyte blocks. You should never do such a thing as you will probably fill up the filesystem and many tools can't cope with such situations. While the ext4 filesystem gets effectively filled up and the command ends, a compressed filesystem is able to compress the stream of zero bytes aggressively.

| true

This is nonsense, as the dd with of= doesn't have any output. You probably meant || true to neutralize the status code of dd but that's not very useful anyway, as individual command failures don't kill the whole script.

One typical use of dd if=/dev/zero is to zero out the whole device, such as /dev/sdb. As noted in another answer, you may want to use shred instead. In both cases you want to refer to the whole system partitions presented in the system as block devices, not the individual files (that may not exist any more). There are lots of books and online resources on linux block devices and manual pages of shred and dd are also very useful.

When you delete a file, its blocks are immediately available. You don't need to perform any black magic. If you want to wipe out the contents of a file to prevent forensic techniques, just use schred --delete to remove the file instead of a normal delete. Wiping out blocks of an already deleted file is not so easy and you cannot simply rely on creating a large empty file. You would need a specialized filesystem specific tool to do it securely.

4
  • I read in various places that you can use dd to zero out the filesystem, that is to reclaim space from deleted files. I'm not trying to wipe out the disk, just reclaim space. Now that it is apparent that doing this in ZFS doesn't work, is there a way to actually reclaim deleted file space inside ZFS then? I noticed you said shred, but in my case I already have deleted files, I cannot delete them again because they are no longer there. Commented Jul 7, 2014 at 22:50
  • The | true is just a trick to use inside a script so that the failure doesn't exit the script. Commented Jul 7, 2014 at 22:51
  • @CMCDragonkai No it isn't, that would be || true, not | true. Commented Mar 16, 2015 at 11:54
  • @CMCDragonkai I don't have any details on ZFS to be honest. But I consider the very technique of filling up a filesystem using a newly created file an attempt to perform black magic. There's no reason to attempt to reclaim an already reclaimed space. Commented Mar 16, 2015 at 12:06
3

You're running into ZFS's compression. If you're trying to wipe out the filesystem, then you can't do it that way in ZFS. If you're trying to wipe out the underlying disk, the best thing to do would be to use "zpool destroy" on the zfs pool, then use dd on the physical device itself. ("dd if=/dev/zero of=/dev/...."). Once that's done, just recreate the pool with zpool create.

1
  • The notes about compression seems to be right but I don' think this answer contains a proper explanation of what's happening. Commented Jul 7, 2014 at 14:56
1

I have no idea about ZFS but if your intention is to zero out the file system then rather dd I will suggest shred

I'm not answering about why dd is in infinite loop but your end-target is solved

1
  • How does shred achieve that, it looks like it just deletes things really well. What if I already have stuff deleted? Commented Jul 7, 2014 at 6:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .