5

How can I generate 10 MB files from /dev/urandom filled with:

  • ASCII 1s and 0s

  • ASCII numbers between 0 and 9

6
  • ASCII numbers between 0 and 9

    < /dev/urandom tr -dc '[:digit:]' | head -c 10000000 > 10mb.txt
    
  • ASCII 1s and 0s

    < /dev/urandom tr -dc 01 | head -c 10000000 > 10mb.txt
    
  • 2
    Note that the second one for instance needs to read (on average) 1280 MB from /dev/uramdom to get 10 MB of output. See pv -cN a < /dev/urandom | tr -cd 01 | pv -cN b > /dev/null – Stéphane Chazelas Aug 19 '14 at 14:02
  • Since /dev/urandom will not block, this isn't a problem, right? – programings Aug 19 '14 at 14:04
  • 3
    @programings, no, just not very efficient compared to < /dev/urandom perl -pe 'BEGIN{$/=\4096}; $_=unpack("b*",$_)' for instance. – Stéphane Chazelas Aug 19 '14 at 14:12
  • @StéphaneChazelas - I don't think the perl solution is even a contender for efficiency vs tr, but I think that deleting the randomized input is probably not a good way to go. – mikeserv Mar 25 '15 at 7:40
  • From man tr: SET2 is extended to length of SET1 by repeating its last character as necessary. This would be really easy if SET2 was repeated instead. – Simon Kohlmeyer Mar 29 '18 at 11:38
5

If you consider that the actual value for each byte you receive from </dev/urandom is only significant in that it represents a successful chance occurrence of that byte's value as determined by the PRNG, then you'll realize that whether or not an input byte matches the value for the ones you're looking for is not nearly as important as how often it does. If the PRNG is any good, then any byte in the ASCII spectrum should have a 1/256 chance of occurring for each byte you read.

If you wish to narrow that spectrum to some ASCII subset, then the most efficient way to handle that is to simultaneously broaden the occurrence chance for those chars in your subset and remove the chances of any other. tr is very good at this, as it allows you to convert chars in a specified range to so many occurrences of a replacement char. Like this:

d=$(printf '[%d*25]' 1 2 3 4 5 6 7 8 9)
</dev/urandom LC_ALL=C tr '\0-\377' "$d[0*]"

There are a few things going on there, and they are:

  1. d=[[char]*[num]]...

    • here I just setup a var containing the second argument I mean to hand tr in the next line. Each [] square-bracketed value is a conversion target for tr and each *25 value represents how many members of a range in order as specified in tr's first argument should target this char for conversion.
  2. LC_ALL=C

    • This (importantly) mandates that every byte read should be interpreted as an ASCII byte, and so all bytes read in will be any of NUL through octal \377.
  3. '\0-\377' "$d[0*]"

    • this instructs tr to convert all input bytes according to the value in $d. This means that the bytes \0-\30 (or the first 25 bytes in the range) are converted to ones, \31-\61 to twos and so on.

The result is that all input is converted to only digits across an (almost) even distribution of randomness - and so every byte is used, but they all wind up being only the ones you want. With the above example, though, there is a 4% greater chance that a 0 will occur in tr's output than any of the other bytes. If this is a problem, you can also do:

LC_ALL=C </dev/urandom \
tr '\0-\377' "[\0*5]$d[0*]" | 
tr -d \\0

...which resolves that issue.

Now, for the 10M thing, this will work:

TR PIPELINE | dd bs=4k count=2560

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.