1

I am using @tremby's great idea to fill a disk with random data.

This involves piping openssl, which is encrypting a lot of zeros, to dd (bs=4M).

I'm maxing out the single core on which this is being run (I have 7 more), and I'm nowhere near maxing out my I/O.

I'm looking for a way to parallelize the input to dd.

I suppose I could do it like this, but what I'm really looking for is a way to parallelize openssl and write that to dd so that the write to the disk is sequential.

Does anyone have a suggestion?

  • PLease, take a look at this question and see if it fits on your needs :) - unix.stackexchange.com/questions/160469/how-to-parallelize-dd – user34720 Jan 5 '16 at 19:25
  • Not quite, thanks. I was clarifying the question, but you were too fast for me! :) – Diagon Jan 5 '16 at 19:29
  • Instead of encrypting zeros why not just use random data: openssl rand 123456789 – Ole Tange Jan 5 '16 at 23:07
  • @Ole_Tange - Not sure what's faster. The complaint over at that first link (to which tremby was responding) was that /dev/urandom was too slow; so I was taking it that the openssl method he proposed was faster and just as dependably random. Perhaps this would work just as well? – Diagon Jan 6 '16 at 6:07
  • @Ole_Tange - I tried your suggestion with openssl rand 2trillion > fifo_file &. Once I start reading from the fifo, the openssl process dies after about 16MB. I didn't have through problem with @tremby's approach (linked above). – Diagon Jan 7 '16 at 17:16
2

The important part is having a way to merge the output of your several openssl commands. I believe a FIFO would solve your problem. Try this

mkfifo foo
openssl <whatever your command is> > foo &
openssl <whatever your command is> > foo &
openssl <whatever your command is> > foo &
dd if=foo of=/dev/sda bs=4M

EDIT: Add as many of the openssl lines as you need to max out your system; you can even add them after dd invocation.

As mentioned by the OP in the comments below, it is possible to cat foo | pv | dd of=/dev/sda to monitor progress.

  • Awwwesome! There is no end to learning unix. Thanks. – Diagon Jan 5 '16 at 19:59
  • IMHO named pipes (aka FIFO's) are one of the most under utilized features of Linux. I'm here to spread the good word :) – David King Jan 5 '16 at 20:07
  • can I cat foo | dd ? I ask because if so, I can use pv to see how far I've go to go. – Diagon Jan 5 '16 at 20:08
  • Sure. You can treat foo just like any regular file except that it won't get to the end until the openssl commands terminate. – David King Jan 5 '16 at 20:09
  • 1
    @Dominik - Well, I usually do that because I want to use specific options with cryptsetup (like whirlpool or detached headers). Also, I usually want to install using btrfs with subvolumes set up in some particular way. It's not hard. Just use the live usb, prepare the disks and then run the installer. Then you can install to the prepared disks, without doing any formatting. – Diagon Jan 6 '16 at 10:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.