9

I'm overwriting my hard drive with random data using the good old dd:

dd if=/dev/urandom of=/dev/disk/by-uuid/etc bs=512

It's a 2TB array and my MacBook (running Linux, ok?) can only write data at around 3.7MB/s, which is pretty pathetic as I've seen my desktop at home do 20MB/s. When I go home tonight, I'd like to stop the dd run here, take it home, and see what kind of progress can be made overnight with a more powerful machine.

I've been monitoring the progress using a simple loop:

while true; do kill -USR1 $PID ; sleep 10 ; done

The output looks like this:

464938971+7 records in
464938971+7 records out
238048755782 bytes (238 GB) copied, 64559.6 s, 3.7 MB/s

If I were to resume the dd pass at home, how would I restart it? I'm aware of the seek parameter, but what do I point it to, the record number or the byte count?

  • 1
    Do I use the record number? Does that equal the block written count? – Naftuli Kay Jan 21 '15 at 19:25
  • 2
    The number of blocks = total bytes / block size, in theory it should be 238048755782 / 512 = 464938976 but you have some partial records there so I would subtract a few blocks from the number just to be safe, e.g. seek=464938960 – don_crissti Jan 21 '15 at 19:36
8

As @don_crissti already commented, just use seek= to resume.

dd if=/dev/urandom of=/dev/disk/by-uuid/etc bs=512 seek=464938971

GNU dd also supports seeking in bytes, so you can resume exactly, regardless of blocksize:

dd if=/dev/urandom of=/dev/disk/by-uuid/etc bs=1M \
   seek=238048755782 oflag=seek_bytes

A larger blocksize should help with speeds even for a slow device like /dev/urandom.

If you are looking for faster alternatives, you could cryptsetup plainOpen with a random key and zero that, it should beat /dev/urandom by an order of magnitude (without AES-NI) or even run at full speed (with AES-NI).

You could also use shred -n 1 if pseudorandom data is good enough for your use case. shred should be able to utilize the full disk speed, even on a very slow machine.

  • I didn't know about plainOpen until now. Great! Finished my scrambling of a 2TB drive in around 4 hours as opposed to 256GB in over 12 using /dev/urandom. – Naftuli Kay Jan 22 '15 at 18:15
3

Just a reminder for people who would like to copy rather than just randomizing disks (which is not that common) : you can use skip=BLOCKS to start reading at the proper position, and seek=BLOCKS to start writing at the correct position. Both options use blocks, not bytes. When breaking/restarting, it's advisable to remove a bunch of blocks just in case. It is usually worth raising the bs value above 512, as you can reach better performance if you read a lot of data in a row.

In your case, it is indeed a block value that you need to pass to seek. Maybe you should try to adjust bs to see if you can enhance speed, as /dev/random should go fast (pseudo-random and non-blocking when it has no entropy available)

0

dd with a tiny block size like 512 bytes is likely to be a lot slower than your disk's maximum throughput. Use a higher block size (on a hunch I'd say a few MB) for good performance. Or use cat — on Linux I found cat to be as fast as dd with the optimal block size when a single disk was involved (I don't know if that holds for OSX as well).

To find how far cat has reached, run lsof -p1234 where 1234 is the process ID of the cat process.

To resume from a position, use

{ dd bs=1 seek=123456; cat /dev/urandom; } >/dev/disk/…

where 123456 is the offset in bytes.

0

Cloning a disk:

Expanding on this answer from this thread, this is how one might go about cloning a whole disk and resuming:

This example is optimized for copying from a 5400rpm rotary drive to an SSD on a specific system. gdd represents GNU dd:

> sudo gdd 'if=/dev/rdisk3' 'of=/dev/rdisk6' bs=4M status=progress
247426187264 bytes (247 GB, 230 GiB) copied, 2082 s, 119 MB/s
59012+0 records in
59011+0 records out
247510073344 bytes (248 GB, 231 GiB) copied, 2082.92 s, 119 MB/s

I can resume this in one of two ways:

> sudo gdd 'if=/dev/rdisk3' 'of=/dev/rdisk6' \
bs=4M \
seek=59011 skip=59011 \
status=progress

Or:

> sudo gdd 'if=/dev/rdisk3' 'of=/dev/rdisk6' \
bs=4M \
seek=247510073344 skip=247510073344 \
oflag=seek_bytes iflag=skip_bytes \
status=progress

In the first example, the reason we use 59011 and not 59012, is because 59011 is how many block sized records were fully copied before being interrupted. (records out).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.