[Botan-devel] Small file ciphering speed

Jack Lloyd lloyd at randombit.net
Wed Sep 24 09:34:52 EDT 2008

On Wed, Sep 24, 2008 at 12:36:26PM +0800, Mr Diggilin wrote:
> an sqlite database each containing two fields that need decrypting...
> 1 ms per row means 30 second load time.
> ???200 microseconds per row means a 6 second load time.
> 100 microseconds per row means a 3 second load time.
> Given my competition running 15,000 rows at 5 seconds, the first is
> enough to force surrender, the second is acceptable, and the third is
> much preferable. This is on a core duo (not core 2 duo) 2.16 ghz. Would
> you guess the latter two goals to be attainable on such a processor?

I am not sure about the Core duo specifically, as I have not used one,
but I spent the early morning implementing a Row_Encryptor class along
with a test/benchmark prog. On my Core2 Duo 2.4 Ghz processor using
GCC, it took .4 seconds to generate 15000 records, encrypt them all
(Twofish/EAX), and then decrypt them all and check them against the
originals (this also includes process startup, randomness polling,
etc). On my 2 GHz Pentium 4-M (keep in mind this is an old Netburst,
so probably comparable to a ~1.2 GHz P3 in a lot of ways) running
Windows XP using Visual C++ 2008, it took about 1.2 seconds to do the
same thing, and .9 seconds when I switched to using AES-128.

Of course once disk loads come into play latency can grow quickly, but
Sqlite is supposed to be reasonably fast, so your goal of 3 seconds
seems pretty achievable.

> That sounds good, and I thought would be the way of things. How would I
> do this in practice though? Right now, in order to do an S2K I need a
> salt that is different for every field, do I just not change the salt
> for each cipher operation? Also where what do you recommend for a random
> IV generation?

This is actually completely unnecessary! However this is hard to see
unless you know EAX well. EAX can actually use any arbitrary input as
the IV, because it first processes that value with a PRF anyway! So,
as long as you know the salt will change each time, you can use it
directly as the IV - EAX does not assume anything about the nonce
other that that it does not repeat (per-key; using the same nonce with
multiple different keys is also fine).

> I don't save the IV, but the salt that generates it. Here's the function
> that is called where I put "???100 round s2k":

Aha. You should be able to remove this entirely then, thanks to the
property of EAX described above.

> Sounds good to me. I can get the row id for the db if needed. As I said
> above, I'm not sure what to do about salt, since it's only used in key
> generation. Should I put the row id to use in getting an IV?

Yes you can even use the row id directly as the IV as long as you can
be assured it will not repeat (eg among multiple databases encrypted
with the same key). Otherwise you could use the (databaseid, rowid)
tuple as the IV.

> Also, is there any cleanup I'll have to do on the pipe between messages?
> Can I just write in and read out indefinitely?

No cleanup is needed. As long as you read out the message contents
entirely, Pipe should be zero overhead (there are no queues of null
pointers or zero-length containers growing longer and longer as you
read/write more messages) - this has been been tested up to (IIRC) 5
million messages or so.

You can see the source for my row encryptor here:


More information about the botan-devel mailing list