4

Linux has shred, from the GNU coreutils package, to securely overwrite data in-place when removing files. What is the equivalent on BSD systems (and specifically on macOS)?

18
  • 2
    shred is a as much snake oil on Linux as it is on BSD. Jul 15 at 15:57
  • 1
    @MarcusMüller Still, it is the equivalent on all platforms.
    – Kusalananda
    Jul 15 at 16:01
  • 1
    @MarcusMüller, though shred can be used for just wiping with zeroes, and IIRC it can print stats on the progress, which is useful for wiping full disks, and makes it strictly better (more featureful) than a plain cat /dev/zero > .... Having a volume mapper like LVM underneath shouldn't matter that much, since (I expect) it's very common that there are minimal changes to the mappings during the lifetime of your regular desktop install. Or at least that one seldom moves existing mappings or resizes fs's to be smaller. [...]
    – ilkkachu
    Jul 16 at 8:58
  • 1
    [...] Merely extending an fs shouldn't leave copies of data around, AFAIU. Btrfs and ZFS would be different, though. And then there's the thing that even if repeated overwrites aren't technically necessary, they might be politically required. :)
    – ilkkachu
    Jul 16 at 9:00
  • 1
    @MarcusMüller, yeah, I think I associate cost as a somewhat necessary part of snake oil, as in someone selling useless crap for high prices based on fraudulent claims (while pretty much knowing it's a fraud). I don't see that in shred, since it's not sold for money, but is free to use, and it's not the authors of it promoting it as an do-all wonder tool (I think?), but more like the dozens of web sites which might just be dated or ignorant. Not as useful as some might claim, yeah; cargo cult, maybe; snake oil, not that much in the way I understand it. :)
    – ilkkachu
    Jul 16 at 11:03

3 Answers 3

7

The answer that qorg11 gave is the technically accurate answer to your question (so go and upvote it!).

The reality is that shred is useless. All BSDs, as well as Linux, have modern file systems, where "overwrite a file to actually overwrite the bytes on the storage medium" simply does not happen because it's not how the structure of a file system looks like and how the storage device works.

What shred does is take a file and overwrite its contents with zeros. Great, but many modern filesystems are "copy on write" (i.e., when you change something in a file, you don't modify the same blocks on the storage device, you make a modified copy of the affected block). Apple's APFS is definitely one of these file systems.

Some file systems have "sparsity detection": you write all-zeros, so the file system doesn't write these zeros to disk but just denotes a hole in the file.

All SSDs have write wear balancing: you're really not protecting against an attacker that can access the raw data as stored on the memory chips because all writes to the system are essentially copy-on-writes to evenly distribute the load of writing data across the medium (writing the same cell often leads to ageing and increased error probability; since an SSD has uniform access latency, wildly distributing logical blocks across the physical medium has no downside). On the flip side, modern file system drivers support the TRIM operation (often called "discard"), which signals to the SSD that the data can be zeroed, and the SSD will then quite likely not even read the zeros if being asked for that logical block.

So, as qorg11 says, shred is the equivalent of shred, even on non-GNU systems, if you install GNU coreutils, but an rm works just as well, in reality.

6
  • 3
    Umm, shred does not simply override a file’s content with zeros. It overwrites with random data, and then optionally may overwrite with zeros in a final pass so that you look clean. This relies on overwrites with random data being performed in place and not being optimized away. Of course, that’s not necessarily true - a filesystem might take advantage of any overwrites to optimize block placement - but it’s true on many file systems. Similarly, the file system might squash all but the last block write, even if flushed or synced.
    – Krazy Glew
    Jul 16 at 3:30
  • Of course shred may not handle block remapping in flash, or even on regular discos as a result of errors or funky disco vendor optimizations. // ultimately, there must be primitives that cross all of the levels of abstraction to do safe erase. And at the moment that is easiest if completely erasing a hardware device.
    – Krazy Glew
    Jul 16 at 3:33
  • 7
    All BSDs, as well as Linux, have modern file systems, where "overwrite a file to actually overwrite the bytes on the storage medium" simply does not happen that's not true at all. The most common Linux filesystem nowadays is still ext4 which is journaled. Btrfs, ZFS and F2FS still didn't quite get mainstreamed
    – phuclv
    Jul 16 at 3:50
  • Fedora installations have defaulted to btrfs for quite a while, haven't seen an Android phone with an ext4 data partition in a while. But you're right, ext4 is not COW; and there was even a proposed safe delete patch (which essentially overwrites after delete) for ext4 that never made it into the kernel - because filesystem maintainers deemed it snake oil to have a journal and then claim you made deletion securer by overwriting the data extents. So in the case of ext4, it's clearly the journaling that drives shred ad absurdum, not COW. Jul 16 at 7:33
  • 1
    Ext4 is not a copy-in-write system, only a journaling file system, so that the changes made to a file appear in a log, which might be rolled back; that's not as bad as having a copy, because journals get purged regularly automatically. However, multiple rounds of overwrite are useless. On apfs, yes, that's copy-on-write and you never overwrite the original data when you change a file. No, you don't have to zero your whole drive, you could also use the TRIM operation to tell the drive to discard the data from unused blocks, if an ssd. Jul 18 at 7:13
6

You can install the GNU coreutils with brew install coreutils in macOS (using Homebrew), pkg install coreutils in FreeBSD, or pkg_add coreutils in OpenBSD and then run gshred, which will behave exactly like shred in Linux.

3

rm itself has the -P flag.

OS X man page documents it as:

-P   Overwrite regular files before deleting them.  Files are
     overwritten three times, first with the byte pattern 0xff,
     then 0x00, and then 0xff again, before they are deleted.

OpenBSD man page:

-P    Attempt to overwrite regular writable files before deleting them. Files 
      are overwritten once with a random pattern. Files with multiple links will 
      be unlinked but not overwritten.`

On FreeBSD however it does nothing, but the flag was kept for compatability.

The BUGS section of the OpenBSD man page warns about its limitations: http://man.openbsd.org/rm#BUGS

The -P option assumes that both the underlying file system and storage medium write in place. This is true for the FFS and MS-DOS file systems and magnetic hard disks, but not true for most flash storage.

These also applies to shred.

2
  • This is very good to know. Macos man page states this: This flag has no effect. It is kept only for backwards compatibility with 4.4BSD-Lite2. Jul 18 at 6:41
  • Ah, macOS must have bought the rm inline with freebsd's.
    – carbin
    Jul 18 at 10:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.