19

If I do watch cat /proc/sys/kernel/random/entropy_avail I see that my systems entropy slowly increases over time, until it reaches the 180-190 range at which point it drops down to around 120-130. The drops in entropy seem to occur about every twenty seconds. I observe this even when lsof says that no process has /dev/random or /dev/urandom open. What is draining away the entropy? Does the kernel need entropy as well, or maybe it is reprocessing the larger pool into a smaller, better quality pool?

This is on a bare-metal machine, with no SSL/SSH/WPA connections.

  • Good question, and while I can't give you a specific answer, it does make sense to me that the "entropy pool" is not simply something that should increase when unused. /dev/random is, after all, something that is used for secure cryptographic purposes, and the implementation can't afford to be naive. One explanation might be hinted at in the last point here: en.wikipedia.org/wiki/Entropy_pool#Using_observed_events (starting with "Maintain a stream cipher with a key and Initialization vector...") -> the pool is replaced whenever enough data has accumulated. – goldilocks Oct 20 '13 at 17:04
  • Do note that in any case, Linux's entropy calculation in /dev/random is pretty much bogus — once the entropy pool has been full once, /dev/urandom is just as good as /dev/random. – Gilles Oct 21 '13 at 16:04
  • 1
    @techraf Wow, that answer got big fast. I never expected to get an answer 2.5 years later. – wingedsubmariner Jun 21 '16 at 18:52
18

Entropy is not only lost via /dev/{,u}random, the kernel also takes some. For example, new processes have randomized addresses (ASLR) and network packets need random sequence numbers. Even the filesystem module may remove some entropy. See the comments in drivers/char/random.c. Also note that entropy_avail refers to the input pool, not the output pools (basically the non-blocking /dev/urandom and the blocking /dev/random).

If you need to watch the entropy pool, do not use watch cat, that will consume entropy at every invocation of cat. In the past I also wanted to watch this pool as GPG was very slow at generating keys, therefore I wrote a C program with the sole purpose to watch the entropy pool: https://git.lekensteyn.nl/c-files/tree/entropy-watcher.c.

Note that there may be background processes which also consume entropy. Using tracepoints on an appropriate kernel you can see the processes that modify the entropy pool. Example usage that records all tracepoints related to the random subsystem including the callchain (-g) on all CPUs (-a) starting measuring after 1 second to ignore its own process (-D 1000) and including timestamps (-T):

sudo perf record -e random:\* -g -a -D 1000 -T sleep 60

Read it with either of these commands (change owner of perf.data as needed):

perf report  # opens an interactive overview
perf script  # outputs events after each other with traces

The perf script output gives an interesting insight and shows when about 8 bytes (64 bits) of entropy is periodically drained on my machine:

kworker/0:2   193 [000]  3292.235908:       random:extract_entropy: ffffffff8173e956 pool: nbytes 8 entropy_count 921 caller _xfer_secondary_pool
                  5eb857 extract_entropy (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  5eb984 _xfer_secondary_pool (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  5ebae6 push_to_pool (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  293a05 process_one_work (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  293ce8 worker_thread (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  299998 kthread (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  7c7482 ret_from_fork (/lib/modules/4.6.2-1-ARCH/build/vmlinux)

kworker/0:2   193 [000]  3292.235911:         random:debit_entropy: ffffffff8173e956: debit_bits 64
                  5eb3e8 account.part.12 (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  5eb770 extract_entropy (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  5eb984 _xfer_secondary_pool (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  5ebae6 push_to_pool (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  293a05 process_one_work (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  293ce8 worker_thread (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  299998 kthread (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  7c7482 ret_from_fork (/lib/modules/4.6.2-1-ARCH/build/vmlinux)

...

swapper     0 [002]  3292.507720:   random:credit_entropy_bits: ffffffff8173e956 pool: bits 2 entropy_count 859 entropy_total 2 caller add_interrupt_randomness
                  5eaab6 credit_entropy_bits (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  5ec644 add_interrupt_randomness (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  2d5729 handle_irq_event_percpu (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  2d58b9 handle_irq_event (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  2d8d1b handle_edge_irq (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  230e6a handle_irq (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  7c9abb do_IRQ (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  7c7bc2 ret_from_intr (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  6756c7 cpuidle_enter (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  2bd9fa call_cpuidle (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  2bde18 cpu_startup_entry (/lib/modules/4.6.2-1-ARCH/build/vmlinux)
                  2510e5 start_secondary (/lib/modules/4.6.2-1-ARCH/build/vmlinux)

Apparently this happens to prevent waste of entropy by transferring entropy from the input pool to outputs pools:

/*
 * Credit (or debit) the entropy store with n bits of entropy.
 * Use credit_entropy_bits_safe() if the value comes from userspace
 * or otherwise should be checked for extreme values.
 */
static void credit_entropy_bits(struct entropy_store *r, int nbits)
{
    ...
        /* If the input pool is getting full, send some
         * entropy to the two output pools, flipping back and
         * forth between them, until the output pools are 75%
         * full.
         */

         ...
            schedule_work(&last->push_work);
}

/*
 * Used as a workqueue function so that when the input pool is getting
 * full, we can "spill over" some entropy to the output pools.  That
 * way the output pools can store some of the excess entropy instead
 * of letting it go to waste.
 */
static void push_to_pool(struct work_struct *work)
{
    ...
}
  • 2
    +1 for the pointing out that even seemingly "innocous" operations such as starting a program can drain a small amount of entropy. – a CVn Oct 21 '13 at 12:52
  • This explanation is however a bit contradictory to the situation described in the question, isn't it? There the entropy (monitored with watch) grows steadily then drops sharp. If watch consumed entropy on every reading, it should actually decrease steadily. – techraf Jun 19 '16 at 21:52
  • @techraf Good observation, periodically invoking cat should in theory have the same entropy drain which should not be visible. It turns out that entropy is moved to a different pool when there is "sufficient" entropy. – Lekensteyn Jun 20 '16 at 18:39
4

lsof is not the best tool to monitor /dev/random as one read by a process is over in a very short amount of time. I do not know of a good method of getting what process is doing a read, but using inotify you can monitor if there is a read.

Here there is basically two ways:

  1. Get a summary after N seconds with:

    inotifywatch -v -t 60 /dev/random 
    
  2. View live access events:

    inotifywait -m --timefmt '%H:%M:%S' --format '%T: %e' /dev/random
    

Neither will give you process and the latter will not give you size of read. The first will give you a summary as in:

total  access  close_nowrite  open  filename
18     16      1              1     /dev/random

If you have that running and do a dd if=/dev/random of=/tmp/foo bs=1 count=3, you get the idea.

Anyhow. This will not give you ticks when the kernel consumes from the pool.


When it comes to checking the status of the entropy using

watch cat /proc/sys/kernel/random/entropy_avail

isn't the best idea as each cat is going to consume entropy. (I see now it popped up another answer that also mention this.) I also have some C code for this and tried to locate it yesterday. I'll see if I can find it and update answer later.

  • auditd can log reads from /dev/random (I know there are similar examples on this site). – Gilles Oct 21 '13 at 16:02
  • 1
    What about using the following perl in place of watch cat: use Fcntl 'SEEK_SET'; open(my $fh,"<", "/proc/sys/kernel/random/entropy_avail"); while (1) { print <$fh>; sleep(1); seek($fh,0,SEEK_SET); } – gmatht Mar 6 '15 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.