[Botan-devel] Why Pooling_Allocator?

Jack Lloyd lloyd at randombit.net
Fri Oct 14 18:26:18 EDT 2005


On Thu, Oct 13, 2005 at 08:25:39PM +0800, Matt Johnston wrote:

> I've had a look at this, and I think that the problem lies
> in the exit path of Pooling_Allocator::free_block().  Via
> std::cerr debugging, it appears that real_mem keeps filling
> up with zero length buffers (which have been
> dealloc_block()ed and zeroed), which never get removed from
> the list. This leads to all of the other iterations over
> real_mem being _very_ slow. I've yet to pin down a concise
> testcase to trigger this though.
> 
> The attached patch improves performance considerably, though
> I'm only guessing as to when remove_empty_buffers() is meant
> to be called.

Matt,

I found a small testcase that sets this off nicely (attached), and confirmed
that your patch solves the problem. I inserted some code into
Pooling_Allocator::deallocate which tracked the number of empty buffers in
real_mem; without your patch, the number of empty buffers in real_mem increases
linearly over time. With it, they are removed correctly. I also had to try for
a while to find a testcase - it looks like only certain allocation/deallocation
patterns will cause the leak.

In case you want to try replicating it, the code I inserted for tracking
real_mem in the deallocate function was:

   static u32bit max_unused = 0;
   u32bit this_unused = 0;
   for(u32bit j = 0; j != real_mem.size(); j++)
      if(real_mem[j].length == 0)
         this_unused++;

   if(this_unused > max_unused)
      {
      printf("Unused real_mem buffers: %d\n", this_unused);
      max_unused = this_unused;
      }

Jack
-------------- next part --------------
#include <botan/botan.h>
using namespace Botan;

#include <iostream>

int main()
   {
   try {
      const u32bit SIZE = 64*1024;
      LibraryInitializer init;

      for(u32bit j = 0; j != 100000; j++)
         {
         SecureVector<byte> buf1(SIZE);
         SecureVector<byte> buf2(SIZE);
         SecureVector<byte> buf3(SIZE);
         }
   }
   catch(std::exception& e)
      {
      std::cout << e.what() << std::endl;
      return 1;
      }
   return 0;
   }


More information about the botan-devel mailing list