[Botan-devel] Small file ciphering speed

Jack Lloyd lloyd at randombit.net
Wed Sep 17 23:08:25 EDT 2008

On Thu, Sep 18, 2008 at 08:43:27AM +0800, Mr Diggilin wrote:
> I'm trying to use botan for encryption and decryption of thousands of
> database rows with very little data (~50 bytes) in them. What I'm doing
> currently takes about 1/4th of a second per field, which is *much*
> slower than would make this practical. I have a few thoughts on how to
> optimize, but I thought I'd ask here first to find out:
> a. What's a good way of doing this with Botan?

Some on this below.

> b. What kind of speed can I expect?

Well expectations may vary. :)

Obviously the answer depends on your hardware (almost entirely the
CPU, and to some extent the memory) and the compiler a bit. But for
reasonable commodity hardware (say, anything that one might sanely run
a database process on), I would say anything more than 1 ms is far too
long, and on a decent machine I would expect an operation of this sort
to take tens of microseconds or so, if reasonably optimized; I'm not
talking about heroic measures here, just using the right algorithms, a
bit of caching, and avoiding excessive dynamic memory allocation.

> c. I'm using an S2K with 100 hash rounds for each operation, which isn't
> much. Would there be any security concerns if I reused the same key
> (with more rounds) over all of the thousands of entries?

No, as long as you use different random IVs each time, using the same
key over multiple messages is fine, and is the normal way to approach
this problem. I would say do the S2K on the passphrase once, with many
iterations (100 is tolerable but 1000-10000 is much better - and the
whole idea is you only do it once, when you read the passphrase in
from the config file or arguments, then it has minimal cost overall to

> auto_ptr<Pipe> Crypto::RunCipher(string Passphrase, wxInputStream * In,
> SecureVector<byte>& Salt)
> {
>   Cipher_Dir Dir = (Salt == NULL) ? ENCRYPTION : DECRYPTION;
>   KeyAndIV = 100 round s2k;
>   SymmetricKey Key(KeyAndIV, KeySize); //256 key
>   InitializationVector IV(KeyAndIV + KeySize, IVSize); //128 IV
>   auto_ptr<Pipe> Out???(new Pipe(get_cipher("Twofish/EAX", Key, IV,
> Dir)));
>   Out->start_msg();
>   *In >> *Out;
>   Out->end_msg();
>   return Out;
> }


- What is the Salt? Does it change every time, or is it specific to
a particular database row?

- I see you are generating a random IV here: how are you saving it to
decrypt later?

- For optimization, you want to make this two distinct operations:
  1) Taking a passphrase and a salt and turning it into a key (PBKDF2)
      - this is slow (to make password guessing attacks harder)
  2) Encrypting lots of fields very quickly using said key, and a salt
     (or, better, a persistent row id, if that is feasible)
       - fast because it needs to be

- Saving the Pipe (along with the cipher object, the precomputed key
  schedule, etc) will save a LOT of time in this. You are pretty much
  worst case scenario for the current style of Pipe per message, since
  you only get to process ~50 bytes / Pipe instance. Also, for
  instance each time you call get_cipher to get a new object, it also
  reruns the Twofish key schedule, which is fairly complex itself
  (roughly takes as long as encrypting 16-32 bytes - again, almost as
  much as your whole starting message! Lotta overhead). And
  dynamically allocating the Pipe and returning it in the auto_ptr
  will probably not be fast either (that is an interesting style
  though - is that so callers can capture the Pipe if desired, with it
  being released otherwise?).

  So what you really want to do is when you first realize what the
  passphrase is, run PBKDF2, set up the Pipe with Twofish, etc, and
  just keep it around for each message you want to encrypt. This
  amortizes the S2K, and the Pipe creation costs, over all messages,
  which should easily give a speedup of an order of magnitude.

  Or if you need to support multiple simultanious passphrases you
  could use a map<string, Pipe>

I hope this helps, I can't give perfectly clear answers to all points
since I don't know the value/purpose of the salt and probably numerous
other important bits of info, but hopefully this should get you
started on optimizing.


More information about the botan-devel mailing list