[Botan-devel] clearing pipe messages

Jack Lloyd lloyd at randombit.net
Wed Dec 14 11:21:25 EST 2005

You're exactly right here; if you're handling multiple messages through a
single Pipe, you have to handle keeping track of the message number you're on
(which is typically as easy as calling pipe.message_count() and subtracting

Currently MAX_MESSAGES is defined as as 0xFFFFFFF0; on most current machines,
Pipe will give you serious problems long before that; just holding the (empty)
message queues for that many messages will take at least 32 gigabytes, and that
is not considering the overhead of std::vector (which is probably fairly
minimal, but still nonzero).

There is no way to reset the message number; the best way to handle this
without modifying Botan would be to have a wrapper class that periodically
(say, after 2^16 messages) destroys and reallocates the Pipe object it is
holding, and otherwise just forwards everything to the Pipe. The most obvious
place where you could insert code that deleted the messages and reset the
message counter would be Pipe::destroy, if you wanted to carry a modified

Looking at it, maintaining these structures seems like needless overhead, and
removing it may end up being an interesting problem. I have a couple of ideas
that I'll try out tonight, so this problem may be removed in the near
future. It's easy to simply destroy the messages, but it seems a little
trickier to also get

automatic operation - all existing application code benefits from the change


compatibility - current application code behaves as expected (there are some
                nice corner case in here, so it's not as easy as it seems)

both of which seem like necessary parts of the problem. Once the scaling issue
is solved (which I think it will - I have two different methods in mind right
now that would _work_, I'm just not sure how clean they will be to implement or
how fast they will be (yet)), that does still leave the original problem with
MAX_MESSAGES; however, most of the Pipe code is structured so that it would be
pretty easy to change the message number to a 64-bit value. Right now there is
no point in doing so, because the aformentioned overhead will drop your process
long before you hit 2^32 messages.


On Wed, Dec 14, 2005 at 04:22:43PM +0500, Muzaffar Mahkamov wrote:
> Hi Jack,
> I think I've found the problem. read() operation increments the
> message number and the next time i was attempting to read the default
> message (which is empty) and the new message remained in the pipe. If
> I increment the message number and pass it to read() operation
> everything works fine. On the other hand, message number may reach
> Botan::Pipe::MAX_MESSAGES in my case. Is there a function to reset the
> message number or how could I do that without side effects by tweaking
> the Botan code? 
> Muzaffar

More information about the botan-devel mailing list