[Botan-devel] Desiring comments on RNG design changes

Jack Lloyd lloyd at randombit.net
Tue Jun 24 15:47:01 EDT 2008

I'm contemplating some design alternatives/problems, and thought I
would see if anyone had suggestions/comments.

So, as background, in 1.7.7 the global RNG object is (hopefully) going
away completely. That's mostly accomplished already (but not quite
yet, hoping to get that part completed and 1.7.7 released sometime in
July). Where the global RNG was used, instead the user has to pass in
an RNG object (a RandomNumberGenerator&, to be specific). This is good
for a few reasons:

  - It makes it much easier to test (previously the test suite did
    hackish things like replace the RNG object, run some tests that
    relied on a certain fixed RNG output, then replaced it back with a
    real RNG). Now it just passes the fixed-output RNG object directly
    to the operation being tested.

  - It reduces thread contention. Obviously as a global resource, the
    global RNG has to be locked. This requires overhead that is either
    (in single threaded programs) unnecessary or that serializes
    threads, reducing the level of concurrency (and thus, presumably
    performance) in multithreaded programs. Now each thread can
    maintain its own RNG object (or the application can maintain an
    app-wide RNG that uses whatever locking scheme it prefers).

  - It minimizes data races. Frankly MT-safe code is hard to write (at
    least for me, maybe there are people who can write 100% correct
    concurrent shared memory code but I am not among them), and it
    seems like the easiest way to do it is avoid mutable shared
    state. So where that is feasiable I'm trying to point Botan in
    that direction (viz botan.remove-libstate branch, etc).

OK. So taking the global PRNG going away as a given, how does the
application create an RNG to use? We could make each application
figure out which entropy sources were available, create a Randpool,
maybe pass the Randpool pointer to an X9.31 PRNG, seed the RNG,
etc. However that seems silly for multiple reasons (code duplication
across projects (nearly any code using Botan would have similar code),
bug prone (hard to write, and if you do it wrong your RNG is broken
which is not good), not extensible (ideally adding a new entropy
source to the library would be easy and applications would benefit
with just a recompile at most)).

Solution? New interface
   RandomNumberGenerator* make_rng();
[As a static RandomNumberGenerator method? As freestanding? Other?]

which is implemented as something like:

   RandomNumberGenerator* rng = new ANSI_X931_PRNG("AES-256", new Randpool);
   std::vector<EntropySource*> sources;
   // collect list of available EntropySources into sources
   //   [code from Builtin_Modules::entropy_sources()]

   // seed rng with sources?
   return rng;

So what I'm contemplating but am unsure about:

Does it make sense to remove the EntropySource handling entirely from
the base RNG class and just have that be more or less and Randpool
specific thing? For instance, have the Randpool maintain a
std::vector<> of the entropy sources; periodically (using some logic
only known to it), it would poll some or all the sources and use it to
add entropy to its internal state. So the //...s above would be

randpool.add_entropy_source(new Device_EntropySource(/*args*/));
// etc

With no explicit seeding step; when the Randpool is first asked for
output it would realize it had not been seeded and run a poll.

Pro: This lets make_rng() return a singular object that will just work

Pro: Applications don't have to know about/configure entropy sources

Pro: Randpool can repoll for more entropy when it wants to (which
     could give it /dev/random like semantics where it blocks to
     repoll if it thinks it does not have enough entropy in its pool
     (I'm not sure if that is a good idea or not, but certainly
     automatic repolls would be nice for long-lived apps).

Con: Makes it hard to add application-specific entropy data (does
     anyone do this, though?). Also makes it hard to use
     application-specific entropy source classes (again, does anyone
     do this?)

Con: Maybe causes too tight a cohesion between Randpool and
     EntropySource (?)

Con: It's not clear how one signals a top level RNG sitting on top of
     a Randpool that it should reseed itself. Currently X9.31 will do
     it anytime new randomness is added to the pool. But when the
     Randpool itself is doing the repolling, that indicator goes away.


More information about the botan-devel mailing list