Update #5: JEP 123: SecureRandom Draft and Implementation.

[Brad Wetmore]

Hi Florian,

 > I wonder if this change to src/share/lib/security/java.security-linux
 >
 > -securerandom.source=file:/dev/urandom
 > +securerandom.source=file:/dev/random
 >
 > causes the return of the blocking behavior.

Welcome to my "can of worms." [1]  I hope everything I've said below is 
correct, and haven't made any typos!

 >  In the past, I saw
 > /dev/random-related blocking during server start-up because too many
 > SecureRandom instances needed seeding.  If I follow the code
 > correctly,

If you came to this code fresh and understood it all in 7 hours, I would 
either give you a medal, or hand in my badge.  :)  It's really twisted 
stuff.

 > seeding of non-strong generators now uses /dev/random again, which is
 > subject to blocking.

Given the long history of the JDK and concerns about backwards 
compatibility, our SecureRandom implementations have become *REALLY* 
hard to understand.  Trying to make sense of all this, and now cleaning 
it up, was actually more than 1/2 of the project.  It took me a couple 
weeks of intense code scrutiny/archeology, and then needed to write a 
wiki page just to keep my head wrapped around it.  Likely the most 
confusing issue I've ever worked on.

In a nutshell, the current implementation of SHA1PRNG does indeed read 
from /dev/random while seeding itself and supplying bytes via 
generateSeed(), which is in opposition to the documented behavior of the 
security properties:  Remember the infamous "file:/dev/./urandom" 
workaround for SHA1PRNG? [2]  (sigh...)

I'll try to simplify as much as I can.  I'll be including parts of this 
for the JDK 8 Oracle's Provider Impl page.

In the current environment, here's our current list of providers, in 
default preference order.

Solaris:
PKCS11/NativePRNG/SHA1PRNG

Linux/Mac:
NativePRNG/SHA1PRNG

Windows:
SHA1PRNG/Windows-PRNG

As you probably know, "new SecureRandom()" obtains the first 
implementation in the list, unless you call 
"SecureRandom.getInstance("alg") or SecureRandom.getInstance("alg", 
"provider"), which gets a specific algorithm, either most preferred 
version or by a specific provider, respectively.

The securerandom.source/java.security.egd currently only affects 
SHA1PRNG, but there's a slight wrinkle in that certain property values 
promote NativePRNG ahead of SHA1PRNG in the list of algorithms in the 
SUN provider.  More on this later.

Current behavior:
=================

NativePRNG:
-----------
does not use securerandom.source/java.security.egd at all

engineGenerateSeed: always reads from /dev/random
engineNextBytes:    always reads from /dev/urandom

SHA1PRNG:
---------
Since this is a real PRNG [3], SHA1PRNG requires a very strong seed 
(initial) value, as strong as possible.  If not initially seeded by the 
user via setSeed, SHA1PRNG will call SeedGenerator.getSystemEntropy + 
SeedGenerator.generateSeed() to generate a strong seed for itself.

There are three seed generators which have SeedGenerator as their parent:

     ThreadedSeedGenerator was the original Seeder which uses
     system load to generate seed values, and is only used as a
     last resort.  (SLOW!)

     URLSeedGenerator takes a URL and reads from it.

     NativeSeedGenerator uses /dev/random.

When the SeedGenerator class initializes, if the properties 
(securerandom.source/java.security.egd) resolve to either the exact 
strings "file:/dev/random" or "file:/dev/urandom", it will use 
NativeSeedGenerator (/dev/random).  (Yeah, try explaining that to 
customers!)  If the properties resolve to any other URL, then we use 
URLSeedGenerator with that URL.  Note, this is why the infamous 
workaround "file:///dev/urandom" or "file:/dev/./urandom" works.  [2] 
As someone once said, this has got to be the most confusing workaround 
in the history of the JDK.  I would agree!  :)  If neither SeedGenerator 
is available, then we use ThreadedSeedGenerator as the last resort.

Once seeded, then later calls to this SHA1PRNG SecureRandom instance are:

engineGenerateSeed: always reads from SeedGenerator
engineNextBytes:    Number generation using the current state as
                     input into SHA1.

So even though the EGD properties point to file:/dev/urandom, it's not 
correct when it comes to the seeder.

Back in JDK 5, adding NativePRNG was the "workaround" for this problem.

When setting up the default order of implementations in the SUN 
provider, if the properties URL is "file:/dev/urandom", then NativePRNG 
becomes the most preferred provider (over SHA1PRNG).  So for those Linux 
implementations which just asked for the "most preferred" (i.e. new 
SecureRandom()), they get an instance of NativePRNG which uses 
/dev/urandom for nextBytes.  (Note well: 
NativeSeedGenerator.generateSeed() still reads from /dev/random).

So since you're not seeding a SHA1PRNG, using NativePRNG.nextBytes() 
makes it appear as though there is no more stalling, as long as you only 
call nextBytes() and not generateSeed().  But if you specifically asked 
for SHA1PRNG (which a lot of customers had baked into their code), you 
still go through the above, and will stall when SHA1PRNG tried to seed 
itself.

(Again, try to explain this to customers.  We've had many escalations 
over the years!)

PKCS11/Windows-PRNG:
--------------------
Didn't touch.  Their behavior doesn't allow have the concept of a EGD.


New Behavior:
=============

So, what do my change do?

.  Corrects the java.security files to reflect the new implementation, 
and corrects vagueness/long-standing errors.  In particular, the 
security property default becomes "file:/dev/random" which matches the 
old AND new implementations, which is what triggered this long discussion!

.  SUN provider:  If the properties are *EITHER* "file:/dev/random" or 
"file:/dev/urandom", then it promotes NativePRNG ahead of SHA1PRNG in 
the list of preferred impls.  If it's not one of those, then SHA1PRNG is 
preferred over NativePRNG.

.  NativePRNG:  Now respects the seeder properties.  If the property 
isn't "file:/" or doesn't exist, we'll fall back to /dev/random.

.  Adds blocking and nonblocking variants of NativePRNG: 
NativePRNGBlocking/NativePRNGJNonBlocking.  The 
generateSeed()/nextBytes() methods both use /dev/random or /dev/urandom, 
respectively.

.  NativeSeedGenerator:  essentially, this becomes the same as a 
URLSeedGenerator.  Instead of only using "/dev/random", it will the 
specified value of "file:/dev/random" or "file:/dev/urandom"

.  Adds SecureRandom.getStrongSecureRandom.  Uses a security property to 
specify the strongest impl(s) on this system.  See the API for more details.

.  Minor style cleanups

I hope this is clear.

Brad

[1] https://www.google.com/search?q=define%3A+can+of+worms

[2] 
http://security.stackexchange.com/questions/14386/what-do-i-need-to-configure-to-make-sure-my-software-uses-dev-urandom

[3] http://en.wikipedia.org/wiki/Pseudorandom_number_generator