Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I often use the command

cat /dev/urandom | strings --bytes 1 | tr -d '\n\t ' | head --bytes 32

to generate pseudo-random passwords. This doesn't work with /dev/random.

Specifically

  • cat /dev/urandom | strings --bytes 1 | tr -d '\n\t ' produces output
  • cat /dev/random | strings --bytes 1 produces output
  • cat /dev/random | strings --bytes 1 | tr -d '\n\t ' does not produce output

NB: When using /dev/random you may have to wiggle your mouse or press keys (e.g. ctrl, shift, etc.) to generate entropy.

Why does the last example not work? Does tr have some kind of large internal buffer that /dev/urandom fills quickly but /dev/random doesn't?

P.S. I'm using CentOS 6.5

cat /proc/version
Linux version 2.6.32-431.3.1.el6.x86_64 (mockbuild@c6b10.bsys.dev.centos.org) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Fri Jan 3 21:39:27 UTC 2014
share|improve this question
    
what is your distribution , your kernel version ? on Cygwin both return values. –  Kiwy Feb 12 at 12:48
    
@Kiwy See edit. –  Aaron J Lang Feb 12 at 13:03
1  
You do know pwgen, in particular pwgen -s? –  MvG Feb 12 at 16:35
2  
The -s switch makes them less memorable, more truly random. @Boyd: is makepasswd widely available beyond the Debian-based distros? The way I see it, pwgen is available for CentOS while makepasswd is not. –  MvG Feb 12 at 17:58
1  
@BoydStephenSmithJr. I concur with @ MvG that makepasswd isn't available on my platform, thanks anyway –  Aaron J Lang Feb 12 at 18:09

3 Answers 3

up vote 14 down vote accepted

It will eventually.

In:

cat /dev/random | strings --bytes 1 | tr -d '\n\t '

cat will never buffer, but it's superfluous anyway as there's nothing to concatenate here.

< /dev/random strings --bytes 1 | tr -d '\n\t '

strings though, since its output is not longer a terminal will buffer its output by blocks (of something like 4 or 8kB) as opposed to lines when the output goes to a terminal.

So it will only start writing to stdout when it has accumulated 4kB worth of characters to output, which on /dev/random is going to take a while.

tr output goes to a terminal (if you're running that at a shell prompt in a terminal), so it will buffer its output line-wise. Because you're removing the \n, it will never have a full line to write, so instead, it will write as soon as a full block has been accumulated (like when the output doesn't go to a terminal).

So, tr is likely not to write anything until strings has read enough from /dev/random so as to write 8kB (2 blocks possibly much more) of data (since the first block will probably contain some newline or tab or space characters).

On this system I'm trying this on, I can get an average of 3 bytes per second from /dev/random (as opposed to 12MiB on /dev/urandom), so in the best case scenario (the first 4096 bytes from /dev/random are all printable ones), we're talking 22 minutes before tr starts to output anything. But it's more likely going to be hours (in a quick test, I can see strings writing a block every 1 to 2 blocks read, and the output blocks contain about 30% of newline characters, so I'd expect it'd need to read at least 3 blocks before tr has 4096 characters to output).

To avoid that, you could do:

< /dev/random stdbuf -o0 strings --bytes 1 | stdbuf -o0 tr -d '\n\t '

stdbuf is a GNU command (also found on some BSDs) that alters the stdio buffering of commands via an LD_PRELOAD trick.

Note that instead of strings, you can use tr -cd '[:graph:]' which will also exclude tab, newline and space.

You may want to fix the locale to C as well to avoid possible future surprises with UTF-8 characters.

share|improve this answer
    
wow impressive explanation. –  Kiwy Feb 12 at 13:06
1  
Amazing! Great explanation and solution. I've always used cat 'uselessly' because I never liked redirecting stdin at the end of a pipeline, now I can 'save a process' and still have readable commands. My final solution was < /dev/random stdbuf -o0 tr -Cd '[:graph:]' | stdbuf -o0 head --bytes 32 –  Aaron J Lang Feb 12 at 13:53
    
@AaronJLang, good point about [:graph:]. I had forgotten about that one. –  Stéphane Chazelas Feb 12 at 14:08
    
@AaronJLang, you don't need the stdbuf for head -c32 unless you want to allow it to write the data as soon as it's got it (like in several chunks instead of one 32byte chunk as soon as it's got them) –  Stéphane Chazelas Feb 12 at 14:50
1  
In my opinion, /dev/urandom is amply sufficient for the author's use. If someone were curious as to how, specifically, urandom works compared to random, I'd suggest reading the comments at the top of the driver in drivers/char/random.c of the kernel source tree. The comments mention an analysis of the PRNG and it's implementation. Hopefully, this paper will answer the question "just how more or less random is urandom compared to random?" Available here: eprint.iacr.org/2012/251.pdf –  etherfish Feb 12 at 18:49

You should use /dev/urandom to get high-quality (pseudo)random numbers, and /dev/random only when you absolutely need random numbers that are really unpredictable. An attacker below NSA's resources will have a very hard time to crack /dev/urandom (and don't forget about rubber hose cryptography). The kernel fills a buffer with "really random" bytes, that is what /dev/random gives. Sadly the rate at which those are generated is low, so reading a lot from from /dev/random will stall waiting for randomness.

You might consider using random.org or its password generator, or one of the many, many random password generators that are floating around, take a look at e.g. this page for a few command line tips (not that I'd recommend all of them, but they should give you ideas), or you could use something like mkpasswd(1) (here on Fedora 19 part of expect-5.45-8.fc19.x86_64).

share|improve this answer

Generating random numbers for many security applications requires sufficient entropy — entropy measures how unpredictable the randomness is. A deterministic processor cannot generate entropy, so entropy must come from the outside — either from a hardware component with non-deterministic behavior, or from other factors that are sufficiently hard to reproduce such as the timing of user actions (that's where wiggling the mouse comes in). Once sufficient entropy is available, cryptography can be used to generate a practically unlimited stream of random numbers.

Linux works by accummulating entropy in a pool, then using cryptography to produce acceptable random numbers both through /dev/random and /dev/urandom. The difference is that /dev/random applies an extremely conservative entropy calculation that reduces the estimate of the entropy in the pool for every byte that it generates, whereas /dev/urandom does not concern itself with the amount of entropy in the pool.

If the estimate of entropy in the pool is too low, /dev/random blocks until more entropy can be accumulated. This can severely cripple the rate at which /dev/random can produce output. This is what you're observing here. It has nothing to do with tr; but strings reads output with buffering, so it has to read a full buffer (a few kB) from /dev/random just to produce at least one byte of input.

/dev/urandom is perfectly acceptable for generating a cryptographic key, because entropy does not in fact decrease in any perceptible way. (If you keep your machine running for longer than the universe has existed, you can't neglect these considerations, but otherwise you're good.) There is only one case where /dev/urandom is not good, which is on a freshly installed system that hasn't had time to generate entropy yet, or a freshly booted system that boots from read-only media.

Elimitating strings from your boot chain will probably speed up your process:

</dev/random LC_ALL=C tr -dc '~-!'

But you can use /dev/urandom here, as long as you take care not to generate passwords on a system that hasn't had time to accummulate sufficient entropy. You can check the level of Linux's entropy pool in /proc/sys/kernel/random/entropy_avail (if you use /dev/random, the figure in this file will be conservative, possibly very much so).

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.