[Botan-devel] Initializing Botan lazily

Jack Lloyd lloyd at randombit.net
Sun Oct 7 20:08:26 EDT 2007

I just checked in a change that will initialize the library's internal
state lazily as needed if it hasn't otherwise been created. You can
not initialize the library, and it will simply do it for itself when
you first request an operation that needs it (if ever - this allows
you to avoid startup costs unless you actually them). Then you call
LibraryInitializer::deinitialize() (which is a harmless and fast
operation if the library was never initialized) just before main()
returns to release library state.

However I'm not sure how to get options to this lazily created
Library_State. That means it uses the default (not thread safe) mutex
type, so this can't be used in threaded programs right now. And you
can't disable the selftest, which in one usage model (see the commit
message that follows for details) is somewhat costly computationally.
Does anyone have suggestions on this?

The patch is simple:

# old_revision [d9acb02ac838cf503b190f0ffd73c645ec76ab15]
# patch "src/libstate.cpp"
#  from [ff89a1505fed854b0dce45db7b7ae802706785f4]
#    to [6b5f66445f9221b5e16cd1eac05fbae2c079de1c]
--- src/libstate.cpp    ff89a1505fed854b0dce45db7b7ae802706785f4
+++ src/libstate.cpp    6b5f66445f9221b5e16cd1eac05fbae2c079de1c
@@ -31,7 +31,7 @@ Library_State& global_state()
 Library_State& global_state()
-      throw Invalid_State("Library was not initialized correctly");
+      LibraryInitializer::initialize();
    return (*global_lib_state);

And here's the checkin message:

o   -----------------------------------------------------------------
|   Revision: 7924dcfd3cf52e11c00ced9ce45bdf7c3bc42339
|   Ancestor: d9acb02ac838cf503b190f0ffd73c645ec76ab15
|   Author: lloyd at randombit.net
|   Date: 2007-10-07T22:53:59
|   Branch: net.randombit.botan
|   Modified files:
|           src/libstate.cpp
|   ChangeLog:
|   If we attempt to access the global state, and it is null, call
|   LibraryInitializer::initialize(), which will set it for us (or fail by
|   throwing an exception, which will be propogated to the caller). So any
|   instances of creating a LibraryInitializer where no option arguments
|   are passed can be removed; instead that initialization will run when
|   or if you execute an operation where Botan requires the services
|   provided in the state. Because no options are passed, the library will
|   be using the default (debug and not thread safe) mutex type: so
|   hopefully you'll quickly get an exception when the debug mutex
|   realizes it is being used in a threaded application, but there is risk
|   of operations silently failing before that happens.
|   You can call LibraryInitializer::deinitialize() at the end of your
|   main function (or whenever you think you won't need Botan anymore), to
|   free the global state; if not a number of cleanup destructors will not
|   run (including the final scrub of memory).
|   You can even shut down Botan speculatively; if it turns out you need
|   it again, it just means you'll have to take the cost of another
|   initialization. However in applications that use Botan only in small
|   bursts, or in rarely taken codepaths, you can remove the state
|   entirely and suffer zero memory overhead. This probably only makes
|   sense in memory constrained systems, but it's reasonable to do now.
|   Speculatively deallocating the state is probably not thread safe
|   without extra work. One thread calling deinitialize() would invalidate
|   pointers that would have been visible to other threads. One (untested)
|   idea: have an atomic integer with the number of current threads using
|   Botan. If any thread decrements and hits zero, it could deinitialize
|   Botan safely. This might cause too many repeated startup/shutdowns,
|   which would depend on the app use pattern.
|   In addition, since you can't pass arguments to the new Library_State,
|   you can't specify the use of real mutexes (or anything else): so for
|   right now, this only works in applications that are fine with the
|   standard options. I want to find a way to get that working, though,
|   since it's very inelegant. Currently a Default_Mutex (not at all
|   thread safe but somewhat error checking) will be used. And self test
|   will always be run (more on that below).
|   I wrote a program that just initializes and shuts down in a tight
|   loop. Running on my Gentoo box (Core2 E6400, gcc 4.1.2):
|   thread_safe?   selftest?     time (ms)
|   ------------   ---------     ---------
|   no             yes           6.1
|   no             no            3.8
|   yes            yes           6.7
|   yes            no            3.8
|   If you're actually worried that the library might start up OK but then
|   start failing basic self tests, what you actually want to do is have a
|   thread that runs diagnostics on your entire process state (including
|   calling Botan's self test code) every N seconds.
|   The question is how to get arguments from the outside world to the
|   constructor of the Library_State that is created inside of
|   global_state(): avoiding many self tests to save a bit of time (many
|   applications won't care about the extra cost but sometimes 2 or 3 ms
|   is important), and thread safety (beacuse you can't specify to use a
|   real mutex).

More information about the botan-devel mailing list