diff options
author | Rob Austein <sra@hactrn.net> | 2019-04-15 16:40:39 +0000 |
---|---|---|
committer | Rob Austein <sra@hactrn.net> | 2019-04-15 16:40:39 +0000 |
commit | 5decc49d4672d29c6b91fc01825b9dbef074b00a (patch) | |
tree | 46c400f99b939c0c95764f1b5f7851ba177857e5 | |
parent | b0f2c8c1c607648b11a5499a425bc87c0bd6fa22 (diff) |
Add missing README and copyright
-rw-r--r-- | README.md | 99 | ||||
-rwxr-xr-x | baiji | 14 |
2 files changed, 113 insertions, 0 deletions
diff --git a/README.md b/README.md new file mode 100644 index 0000000..be3692a --- /dev/null +++ b/README.md @@ -0,0 +1,99 @@ +baiji +===== + +Baiji is a package building tool for Debian environments. The overall +design borrows heavily from an existing tool called `whalebuilder`, +which in turn is in effect a reimplementation of `pbuilder` using +Docker to do the heavy lifting. + +Clean room build environment +---------------------------- + +`pbuilder` is one of several tools all of which have the same basic +goal: building packages in an environment specified by the package's +`control` file with minimal dependencies on the native environment of +the machine where the build is being run. + +`pbuilder` is probably the oldest surviving example of this kind of +build environment for the Debian universe, and uses a fairly simple +approach: it keeps a pristine build environment in a compressed +tarball, which it unpacks into a freshly-chrooted tree for every +build, thus insuring a predictable environment. The pristine base +environment only includes the "build-essentials" environment that +every Debian package is allowed to assume is always installed: +anything else specified in the `control` file is installed on demand +every time. While a bit finicky to set up, this is pretty simple in +overall concept, and pretty reliable. Unfortunately, it's also pretty +slow. + +`pbuilder` lacks direct support for dependencies on packages not known +to the Debian project, but it does support hook scripts, which is how +we taught it to use the local package collections that our current +build system stores in `/var/cache/pbuilder/local-packages/`. + +If one looks closely at this setup, it turns out to have a lot in +common with Docker containers. The specific mechanisms are different, +but the overall effect is the same: an isolated environment running a +controlled set of packages and scripts independent of the machine on +which it's running. The difference is that Docker has a lot of +support for building and distributing images based on other images. + +There's a lovely little tool called `whalebuilder` which to a first +approximation is a reimplementation of `pbuilder` on top of Docker. +It turns out that `whalebuilder` doesn't do quite what we want, but +it's close enough to serve as a (very) useful model for a new tool + +baiji +----- + +`baiji` is intended to incorporate the most useful features of both +`whalebuilder` and `git-buildpackage`, along with some custom features +such as the ability to load locally-built `.deb` packages directly +into the constructed Docker image. + +Like both `pbuilder` and `whalebuilder`, `baiji` uses a "base image", +which is constructed using the `debbootstrap` package, along with a +few things needed by `baiji` itself. + +Like `whalebuilder` but unlike `pbuilder`, `baiji` also builds +package-specific images for building a particular source package. +While this may sound wasteful, it turns out that it's not: everything +involved in the per-package image construction process is work that +`baiji` would have to do anyway, and by saving the result `baiji` +makes it possible to reuse that image if it turns out that one has to +build the same version of the same source package repeatedly. By +default, `baiji` only generates a new package-specific build image +when the package's version or build dependencies have changed, but the +user can override this to force a new image. + +Also like `whalebuilder` but unlike `pbuilder`, `baiji` performs the +build itself as a non-root user, which is a better fit for the base +Debian model of building packages, and runs with networking turned +off, so that it can catch undeclared dependencies like a package +quietly attempting to `pip install` 25 new packages from PyPi. + +Like `debuild` and the `pdebuild` but unlike `whalebuilder`, `baiji` +can generate the source package from the source tree when needed. + +As mentioned above, `baiji` supports pre-loading local binary packages +directly into a local APT repository within the package-specific +Docker image. + +`baiji`'s base images can be pre-configured to know about local APT +servers,, so that any packages needed for a build can be downloaded +directly. `apt` running within the usual build tools will use the +package version numbers to decide which version of a package to use +when more than one is available. + +`baiji`'s current Interface to Docker just launches the Docker CLI +tool in a subprocess, which turns out to be a lot simpler than using +the official Docker Python API; the latter is only available from +GitHub and PyPi, and depends on enough other PyPi packages that it's a +bit of a maintenance nightmare. Perhaps this will calm down someday, +but for now it's worth the minor inefficiency of using the Docker CLI +to avoid the swamp. + +Unlike `whalebuilder`, which uses uses Ruby's built-in Jinja-like +template mechanism, `baiji` currently just includes the handful of +templates it needs inline using Python's `str.format()` method. Some +day `baiji` may outgrow this, but it suffices for now. @@ -1,5 +1,19 @@ #!/usr/bin/env python +# Copyright (c) 2018-2019, Grunchweather Associates +# +# Permission to use, copy, modify, and/or distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH +# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +# AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, +# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE +# OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +# PERFORMANCE OF THIS SOFTWARE. + import debian.changelog import debian.deb822 import subprocess |