[Botan-devel] RFC: memory locking

[Nathaniel Smith]

On Mon, Nov 12, 2007 at 09:33:14AM -0500, Jack Lloyd wrote:
> 
> Question: Do you use the memory locking abilities of Botan?

I assume by locking you mean mlock(2), not mutexes.

> In general my preference is strongly in favor of using standard
> containers where possible, but previously that hasn't really been
> considered an option due to the desire to support different locking
> strategies.  However, supporting this code has a nonzero cost (not to
> mention that using nonstandard containers implies extra effort people
> to learn the library, that the containers and allocators are not as
> efficient or useful as std::X and the libc heap, and that memory
> allocation is not at all a core mission/goal of Botan and thus not
> something it does particularly well). And I'm not even sure at this
> point if any deployment of Botan actually uses the locking allocators,
> so potentially we're carrying a great deal of extra code/cognitive
> overhead which is being completely wasted. (If the allocators were
> removed, Botan would still zeroize memory as necessary since that is
> trivial to implement portably (though as always you never know what an
> optimizing compiler might do)).

I don't see how you can zeroize a std::vector<> properly; you aren't
there when it gets deallocated, and worse, you aren't there when it
reallocates itself at some moment and leaves behind a copy of
its old contents at some random memory location.  (I don't know why
people would be appending to a std::vector<> representing a crypto
key, of course, but having the possibility in the API is worrisome.

> For the record, I consider locking strategies like this, including
> mlock, as at best a hack to work around operating system
> deficiencies. The only sane thing for an OS to do is zero out swapped
> data pages on process exit.

The only sane thing from a security standpoint, yes; the only sane
thing from a speed standpoint is just the opposite.  Kernels get
pulled on by both camps...

> (Not to mention that swap encrypted with a
> random key chosen at boot time is trivial to implement).

Very true.  But no-one uses it.

I'm nervous about a plan that changes the API in such a way that it
becomes impossible to achieve secure key handling.  In practice, it
might well be impossible to achieve now -- at least without herculean
extra effort (auditing code to be suid safe, for instance, like gpg
does) -- but if those other issues get solved tomorrow, it would be
nice if Botan weren't stuck saying "oops, can't use me".

(At the moment, it looks like the ability to make executables
setu-CAP_IPC_LOCK is going to make it into linux distros before
default-encrypted swap, for instance.)

Of course, by the time C++09 is usable, we'll hopefully know how all
this turned out *anyway* :-).

-- Nathaniel

-- 
In mathematics, it's not enough to read the words
you have to hear the music