Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I am running the following command on an ubuntu system:

dd if=/dev/random of=rand bs=1K count=2

However, every time I run it, I end up with a file of a different size. Why is this? How can I generate a file of a given size filled with random data?

share|improve this question
    
/dev/random will block if there is not enough entropy available to generate the number of digits you want. it simply takes time to gather that amount of high quality psuedo random "randomness"... Either use /dev/urandom for a less random "random" value, or check your entropy pool (in a loop, and wait as need be)... –  Peter.O Feb 28 '12 at 21:18
    
See also this question. –  Keith Thompson Feb 29 '12 at 2:43

3 Answers 3

up vote 19 down vote accepted

You're observing a combination of the peculiar behavior of dd with the peculiar behavior of Linux's /dev/random. Both, by the way, are rarely the right tool for the job.

Linux's /dev/random returns data sparingly. It is based on the assumption that the entropy in the pseudorandom number generator is extinguished at a very fast rate. Since gathering new entropy is slow, /dev/random typically relinquishes only a few bytes at a time.

dd is an old, cranky program initially intended to operate on tape devices. When you tell it to read one block of 1kB, it attempts to read one block. If the read returns less than 1024 bytes, tough, that's all you get. So dd if=/dev/random bs=1K count=2 makes two read(2) calls. Since it's reading from /dev/random, the two read calls typically return only a few bytes, in varying number depending on the available entropy. See also When is dd suitable for copying data? (or, when are read() and write() partial)

Unless you're designing an OS installer or cloner, you should never use /dev/random under Linux, always /dev/urandom. The urandom man page is somewhat misleading; /dev/urandom is in fact suitable for cryptography, even to generate long-lived keys. The only restriction with /dev/urandom is that it must be supplied with sufficient entropy; Linux distributions normally save the entropy between reboots, so the only time you might not have enough entropy is on a fresh installation. Entropy does not wear off in practical terms. For more information, read Is a rand from /dev/urandom secure for a login key? and Feeding /dev/random entropy pool?.

Most uses of dd are better expressed with tools such as head or tail. If you want 2kB of random bytes, run

head -c 2k </dev/urandom >rand

If your head doesn't have a -c option (it's not in POSIX), you can run dd, because /dev/urandom happily returns as many bytes as requested.

dd if=/dev/urandom of=rand bs=1k count=2

In general, when you need to use dd to extract a fixed number of bytes and its input is not coming from a regular file or block device, you need to read byte by byte: dd bs=1 count=2048.

share|improve this answer
    
Thanks for the tip on using head instead of dd. That allows me to still use /dev/random if I want to. Although /dev/urandom would probably be sufficient as you mention, it is nice to know how to use /dev/random should the need arise. –  Daniel Feb 29 '12 at 17:47

Why does dd drop data? ... Gilles has posed this engaging question about dd:
When is dd suitable for copying data? (or, when are read() and write() partial)
Here is an excerpt from that question:

    ...it's not difficult to put dd at fault; for example try this code:*
        yes | dd of=out bs=1024k count=10
    and check the size of the out file (it's likely to be well under 10MB).


Aside from my comment (at the end of your question), something like this is iteresting to watch... It catches your bytes in file $trnd. I've semi-arbitrarily chosen bs=8

Move your mouse and watch it speed up.
With my computer idle (AFK and no Network activity), and after exhausting the entropy pool, it took 2 hours 12 minutes to collect only 1192 bytes, at which point I cancelled it.

Then, with me moving the mouse continuously, it took a relatively much shorter 1 minute 15 seconds to collect the same number of bytes.

This shows pretty clearly that collecting entropy is not CPU speed based, but rather it is random events based, and that my Ubuntu system uses the mouse as one of its significant random factors.

get=2048
trnd=/tmp/$USER.rnd; >"$trnd"
while (( $(wc -c <"$trnd") < $get )) ;do
    dd if=/dev/random bs=8 count=1 2>/dev/null >>"$trnd"
    echo -n "itt: $((i+=1))  ct: "; wc -c <"$trnd"
done
truncate -s $get "$trnd"
echo -e "\nfinal count: "; wc -c <"$trnd"
share|improve this answer

From man 4 random on a RHEL 5 box:

When read, the /dev/random device will only return random bytes within the estimated number of bits of noise in the entropy pool.

I get files of size 213 bytes on that machine. Back to man 4 random:

When read, /dev/urandom device will return as many bytes as are requested.

I get 2048 bytes from every invocation of dd if=/dev/urandom of=rand bs=1K count=2

I conclude that the difference is due to how much entropy your machine generates between invocations of dd if=/dev/random ...

share|improve this answer
    
Yeah, practically, unless he's at a real crypto application, @Daniel should use /dev/urandom. But I'm puzzled as to why dd if=/dev/random bs=1K count=2 stops when the entropy pool is apparently drained. From the docs, it should block until there's more entropy, so the dd will write out the file slowly, instead of just dumping out the current pool and exiting. –  cjc Feb 28 '12 at 21:17
    
I wondered about that too, but it's consistent across RHEL, Slackware 13.1 and a pretty current Arch. The RHEL was x86_64, the others were 32-bit. Unfortunately the dd docs are in GNU info format, so I haven't read them all. –  Bruce Ediger Feb 29 '12 at 2:23
    
It's also consistent on Gentoo too. –  Matthew Scharley Feb 29 '12 at 4:09
1  
@cjc: It's because when you call read(fd, mybuf, 1024) on a blocking FD it returns as soon as the underlying device returns some data. If there are 1024 bytes there to be read, it returns that. If there are only 201 bytes, it will return 201. If there are 0 bytes available, it will block until at least one byte becomes available, then return it/them. –  Warren Young Feb 29 '12 at 20:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.