Why does dd from /dev/random give different file sizes?

Question

I am running the following command on an ubuntu system:

dd if=/dev/random of=rand bs=1K count=2

However, every time I run it, I end up with a file of a different size. Why is this? How can I generate a file of a given size filled with random data?

Answer 1

You're observing a combination of the peculiar behavior of dd with the peculiar behavior of Linux's /dev/random. Both, by the way, are rarely the right tool for the job.

Linux's /dev/random returns data sparingly. It is based on the assumption that the entropy in the pseudorandom number generator is extinguished at a very fast rate. Since gathering new entropy is slow, /dev/random typically relinquishes only a few bytes at a time.

dd is an old, cranky program initially intended to operate on tape devices. When you tell it to read one block of 1kB, it attempts to read one block. If the read returns less than 1024 bytes, tough, that's all you get. So dd if=/dev/random bs=1K count=2 makes two read(2) calls. Since it's reading from /dev/random, the two read calls typically return only a few bytes, in varying number depending on the available entropy. See also When is dd suitable for copying data? (or, when are read() and write() partial)

Unless you're designing an OS installer or cloner, you should never use /dev/random under Linux, always /dev/urandom. The urandom man page is somewhat misleading; /dev/urandom is in fact suitable for cryptography, even to generate long-lived keys. The only restriction with /dev/urandom is that it must be supplied with sufficient entropy; Linux distributions normally save the entropy between reboots, so the only time you might not have enough entropy is on a fresh installation. Entropy does not wear off in practical terms. For more information, read Is a rand from /dev/urandom secure for a login key? and Feeding /dev/random entropy pool?.

Most uses of dd are better expressed with tools such as head or tail. If you want 2kB of random bytes, run

head -c 2k </dev/urandom >rand

If your head doesn't have a -c option (it's not in POSIX), you can run dd, because /dev/urandom happily returns as many bytes as requested.

dd if=/dev/urandom of=rand bs=1k count=2

In general, when you need to use dd to extract a fixed number of bytes and its input is not coming from a regular file or block device, you need to read byte by byte: dd bs=1 count=2048.

Answer 2

From man 4 random on a RHEL 5 box:

When read, the /dev/random device will only return random bytes within the estimated number of bits of noise in the entropy pool.

I get files of size 213 bytes on that machine. Back to man 4 random:

When read, /dev/urandom device will return as many bytes as are requested.

I get 2048 bytes from every invocation of dd if=/dev/urandom of=rand bs=1K count=2

I conclude that the difference is due to how much entropy your machine generates between invocations of dd if=/dev/random ...

Answer 3

Why does dd drop data? ... Gilles has posed this engaging question about dd:
When is dd suitable for copying data? (or, when are read() and write() partial)
Here is an excerpt from that question:

    ...it's not difficult to put dd at fault; for example try this code:*
        yes | dd of=out bs=1024k count=10
    and check the size of the out file (it's likely to be well under 10MB).

---

Aside from my comment (at the end of your question), something like this is iteresting to watch... It catches your bytes in file $trnd. I've semi-arbitrarily chosen bs=8

Move your mouse and watch it speed up.
With my computer idle (AFK and no Network activity), and after exhausting the entropy pool, it took 2 hours 12 minutes to collect only 1192 bytes, at which point I cancelled it.

Then, with me moving the mouse continuously, it took a relatively much shorter 1 minute 15 seconds to collect the same number of bytes.

This shows pretty clearly that collecting entropy is not CPU speed based, but rather it is random events based, and that my Ubuntu system uses the mouse as one of its significant random factors.

get=2048
trnd=/tmp/$USER.rnd; >"$trnd"
while (( $(wc -c <"$trnd") < $get )) ;do
    dd if=/dev/random bs=8 count=1 2>/dev/null >>"$trnd"
    echo -n "itt: $((i+=1))  ct: "; wc -c <"$trnd"
done
truncate -s $get "$trnd"
echo -e "\nfinal count: "; wc -c <"$trnd"