[botan-devel] build system

john skaller skaller at users.sourceforge.net
Sat Jan 26 03:32:04 EST 2013


Hi!

I'm wondering if the configuration script can be modified to optionally output
Python data?

To explain, I will show you the Python code I am using to build Botan.

This code is still a hack: I first ran the configure.py script, which
copies all the supported headers and make a build.h. 

Then I used my build process to compile the library.
Whenever a missing header was found I just removed the compile step.
The code below then actually builds a static link library.
However the dynamic link library fails with many missing symbols.

Ideally what I would do is run the configure script from my build system,
and have that generate the lists of things to compile and include files
to copy. Then the build code below can just load that information,
already in Python format. 

Ideally I could call the configure script from Python (rather than
using system() or popen(). My build system already knows the
compiler to use so I don't want Botan making any guesses.

Actually I prefer it didn't do ANY testing of anything.
If there's something processor dependent or platform dependent,
I need to TELL Botan that information. This keeps control of the
configuration in my product's build script.

This is really important when the build platform isn't the host.
For example just because Debian is building on an SSSE3 capable
machine doesn't mean the client installing the package has one.
Ideally I want to eliminate as many dependencies as possible,
even if that makes the code slower. That produces a "reference"
implementation. Providing overrides for specific platforms
to get performance improvements should be considered an
"optimisation" and always optional.

My product is a cross-cross-compiler so it's quite tricky to handle.
There are 4 platforms: 

	build: where the compiler and run time got built
	host: where the compiler is invoked to generate C++
	target: where the C++ compiler is run, or at least targeted
	run: where the final binaries are used

Many packages assume all four of these platforms are identical.
In particular some autodetection of features just doesn't work in this
kind of environment unless this is the case. Eg. if you have to run
a program to detect the variant of CPU you have.

Here's my code:


import fbuild
import fbuild.db
from fbuild.builders.file import copy, copy_regex
from fbuild.path import Path
from fbuild.record import Record

import buildsystem
from buildsystem.config import config_call

   
# ------------------------------------------------------------------------------

def build_runtime(host_phase, target_phase):
    """
    Builds the botan runtime library, and returns the static and shared
    library versions.
    """

    root = Path('Botan-1.10.4')
    path = Path(root/'src')

    headers = (root / 'build' / 'include' / 'botan'/ '*.h').glob() + [root / 'build' / 'build.h']
    internal = (root / 'build' / 'include' / 'botan'/'internal' / '*.h').glob()

    # Copy the header into the runtime library.
    for h in headers:
      buildsystem.copy_to(target_phase.ctx,
          target_phase.ctx.buildroot / 'lib/rtl/botan', [h])

    # Copy the header into the runtime library.
    for h in internal:
      buildsystem.copy_to(target_phase.ctx,
          target_phase.ctx.buildroot / 'lib/rtl/botan/internal', [h])

    bases =  [
      'algo_base',
    	'block',
      'codec',
      'hash',
      'mutex',
      'pubkey',
      'stream',
      'algo_factory',
      #'build-data',
      'constructs',
      'kdf',
      'passhash',
      'rng',
      'utils',
      'alloc',
      'cert',
      'engine',
      'libstate',
      'pbe',
      #'selftest',
      #'wrap',
      'asn1'
      'checksum',
      'entropy',
      'mac',
      'pbkdf',
      'simd'
      #'benchmark'
      'cms',
      'filters',
      'math',
      'pk_pad',
      #'ssl'
      ]

    srcs = []
    for base in bases:
      srcs.extend ((path / base / '*.cpp'). glob())
      srcs.extend ((path / base / '*/*.cpp'). glob())
      srcs.extend ((path / base / '*/*/*.cpp'). glob())
      srcs.extend ((path / base / '*/*/*/*.cpp'). glob())

    bad = [
      path / 'entropy' / 'cryptoapi_rng' / 'es_capi.cpp',
      ]

    bad.extend ( (path / 'entropy' / 'win32_stats' / '*.cpp') . glob())
    bad.extend ( (path / 'entropy' / 'beos_stats' / '*.cpp') . glob())
    bad.extend ( (path / 'filters' / 'zlib' / '*.cpp') . glob())
    bad.extend ( (path / 'filters' / 'bzip2' / '*.cpp') . glob())
    bad.extend ( (path / 'engine' / 'openssl' / '*.cpp') . glob())
    bad.extend ( (path / 'engine' / 'gnump' / '*.cpp') . glob())
    bad.extend ( (path / 'engine' / 'dyn_engine' / '*.cpp') . glob())
    bad.extend ( (path / 'engine' / 'aes_isa_eng' / '*.cpp') . glob())
    bad.extend ( (path / 'cert' / 'cvc' / '*.cpp') . glob())
    bad.extend ( (path / 'mutex' / 'win32_crit_section' / '*.cpp') . glob())
    bad.extend ( (path / 'hash' / 'sha1_x86_32' / '*.cpp') . glob())
    bad.extend ( (path / 'hash' / 'sha1_x86_64' / '*.cpp') . glob())
    bad.extend ( (path / 'hash' / 'md5_x86_32' / '*.cpp') . glob())
    bad.extend ( (path / 'hash' / 'md4_x86_32' / '*.cpp') . glob())
    bad.extend ( (path / 'block' / 'serpent_x86_32' / '*.cpp') . glob())
    bad.extend ( (path / 'block' / 'aes_ni' / '*.cpp') . glob())
    bad.extend ( (path / 'block' / 'aes_ssse3' / '*.cpp') . glob())
    bad.extend ( (path / 'utils' / '*.cpp') . glob())
    bad.extend ( (path / 'utils' / '*/*.cpp') . glob())

    for p in bad: srcs.remove (p)

    includes = [target_phase.ctx.buildroot/'lib/rtl']
    
    macros = []
    static = buildsystem.build_cxx_static_lib(target_phase, 'lib/rtl/botan',
        srcs=srcs,
        macros=macros,
        includes=includes)

    shared = buildsystem.build_cxx_shared_lib(target_phase, 'lib/rtl/botan',
        srcs=srcs,
        macros=macros,
        includes=includes)

    return Record(static=static, shared=shared)

--
john skaller
skaller at users.sourceforge.net
http://felix-lang.org






More information about the botan-devel mailing list