path: root/rpkid/README

$Id$ -*- Text -*-

Python RPKI production tools.

Requires Python 2.5.

External Python packages required:

- lxml, which in turn requires the libxml2 C libraries.

  http://codespeak.net/lxml/

  FreeBSD: /usr/ports/devel/py-lxml
  Fedora:  python-lxml.i386

- MySQLdb, which in turn requires MySQL client and server.  I'm
  testing with MySQL 5.1.

  http://sourceforge.net/projects/mysql-python/

  FreeBSD: /usr/ports/databases/py-MySQLdb
  Fedora:  MySQL-python.i386

- TLSLite, which pulls in other crypto packages.

  http://trevp.net/tlslite/

  FreeBSD: /usr/ports/security/py-tlslite

- Cryptlib is no longer required.

- Eventually I expect that this will require an event-handling package
  like Twisted, but I'm not there yet.

- The testpoke tool (up-down protocol command line test client) and
  testbed tools also uses PyYAML.

  http://pyyaml.org/

  FreeBSD: /usr/ports/devel/py-yaml

We also use a hacked copy of the Python OpenSSL Wrappers (POW)
package, but our copy has enough modifications that it's expanded in
the Subversion tree.  Depending on how this all works out, I may end
up splitting the POW.pkix module out of the POW package and using it
with Cryptlib, as the POW.pkix package is 98% about doing ASN.1 in
pure Python and only 2% about any kind of crypto.



$Revision$

TO DO:

      - Update BPKI model to what was defined in Amsterdam.  This was
	a direct result of security review by Kent and Housley.

	Much of this is now done.  Remaining tasks:

	    Check chain length in received TLS
	    If TLS cert in SQL is EE:
	       EE cert in SQL must be same as EE cert received from TLS

	PRIORITY: Required for pilot (security issue)

	TIME REQUIRED: Two weeks.

	STATUS: Started


      - rcynic handling of RPKI trust anchors needs updating, per
	discussions over previous months of how RPKI trust anchors
	work, how we package them, and how we roll them over. The last
	(TA rollover) is the driver for this.

	APNIC is now proposing a CMS-signed ASN.1 blob containing a
	version number and an RPKI certificate.  Kent and Housley have
	not bought into this yet.  Need to do analysis to make sure
	this is adequate for our needs, if so just use it.  This would
	involve minor changes to rcynic.

	PRIORITY: Required for pilot (usability issue for relying parties)

	TIME REQUIRED: One week.

	STATUS: Not started


      - Publication protocol and implementation thereof. Desirable
	although not strictly required that protocol be agreed upon
	among the RIRs. Tricky bit is making sure that repository
	receives enough information to know whether parent has
	authorized child to use parent's namespace in nesting case; in
	theory this is straightforward but requires careful checking.

	ARIN can't host output of non-hosted RPKI engines without
	this, and that's critical both to the security model as
	discussed with ARIN staff in late 2006, hence this is a
	required capability even for testing.

	PRIORITY: Required for pilot

	TIME REQUIRED: 3-4 weeks for implementation once protocol
	settled, depending on how much of the existing left-right
	protocol design and implementation can be reused.

	STATUS: Started


      - Resource subsetting (req_* attributes in up-down protocol),
	minimal implementation.  Recognize this as correct protocol
	and signal an internal server error if ever used.

	PRIORITY: Required for pilot.

	TIME REQUIRED: Two days

	STATUS: Not started


      - ROA generation code.  First cut at this seems to work and
        output looks right, but this hasn't been tested properly yet
        due to lack of a ROA validation tool.

	For reasons that presumably made sense at the time, the
	left-right protocol for route_origin objects allows ranges as
	well as prefixes, and the SQL for stores everything as ranges,
	which is nice and general...except that ROAs can only hold
	prefixes.  So left-right schema should only allow prefixes,
	and SQL should only store prefixes.

        Current ROA code implements current draft.  Upcoming draft
        will probably change ROA format to remove per-ROA exactMatch
        field and add a per-prefix maxLength field; implementing this
        will require mods to <route_origin/>.

	PRIORITY: Required for pilot

	TIME REQUIRED: One week (remaining)

	STATUS: Started


      - rcynic does not yet handle manifests. This is both a real
	problem (manifests were added to plug a security hole) and a
	user acceptance problem (without manifest support rcynic
	checks old certs that are supposed to fail because they've
	been revoked, resulting in what appear to be spurious errors,
	which just annoy the user).

	PRIORITY: Required for pilot

	TIME REQUIRED: Two weeks.

	STATUS: Not started


      - User validation tool: fetch and validate certs and ROA for a
	prefix that the user wants to accept in a router filter the
	user is building. This probably uses rcynic's output as one of
	its inputs.

	PRIORITY: Required

	DEPENDS ON: ROA generation

	TIME REQUIRED: 1-2 weeks

	STATUS: Not started


      - Make rpkid fully event-driven (async tasking model), except
	for SQL queries. This probably involves the "twisted"
	framework.

	PRIORITY: Required (to implement scalable hosting model)

	TIME REQUIRED: Two weeks.

	STATUS: Not started


      - Error handling: make sure that exceptions map correctly to
	up-down error codes, flesh out left-right error codes. Note
	that the same exception may produce different error codes
	depending on which up-down PDU we're processing (sigh).

	Will require code audit for coherency, which is most of the work.

	PRIORITY: Required

	TIME REQUIRED: Two weeks

	DEPENDS ON: almost everything else, as almost any code change
	can raise new exceptions that we'd need to handle.

	STATUS: Not started


      - db.commit(), db.rollback(), code audit for data integrity
	issues, fix any data integrity issues that turn up. Among
	other issues, need to handle loss of connection to database
	server and other MySQL errors. Need to be careful about
	recovery action depending on whether we had uncommitted
	changes.

	PRIORITY: Required

	TIME REQUIRED (commit and rollback): 3-4 weeks

	TIME REQUIRED (data integrity audit): 1 week

	TIME REQUIRED (fix data integrity): Unknown, depends on code
	audit and results of runtime testing.

	DEPENDS ON: async tasking model rollback.

	STATUS: Not started


      - Test framework for multiple self-instances per engine-instance
	(single self-instance per engine-instance is already done).

	PRIORITY: Required for testing

	DEPENDS ON: Async tasking model.

	TIME REQUIRED: One week

	STATUS: Not started


      - Current TLS code (tlslite) appeared to be flakey under heavy
	use back in November, and doesn't support all the required
	certificate checks out of the box.

	Certificate checker has now been replaced with something based
	on OpenSSL/POW, and the result seems to work.  If the TLS code
	itself is still unstable, best bet would be to replace it with
	a Tls class cloned from the existing POW Ssl class; the
	current Ssl class isn't adaquate either, but there's
	documentation (eg, the O'Reilly OpenSSL book) that explains in
	some detail what this code would need to do.

	PRIORITY: Required for pilot (cert checking is a security issue).

	TIME REQUIRED: 3-4 weeks

	DEPENDS ON: Async tasking model.

	STATUS: Not started


      - Resource subsetting (req_* attributes in up-down protocol),
        full implementation.  Requires expanding SQL child_cert table
        to hold subset masks and rewriting a fair amount of code.

	PRIORITY: Required for full implementation.

	TIME REQUIRED: 3-4 weeks

	STATUS: Not started


      - Performance testing

	STATUS: Not started


      - Clean up rootd.py to be usable in a production system. Most
	urgent issue is handling of private keys. May not need much
	else, as this is not a high-traffic server.

	PRIORITY: Highly desirable (not strictly needed for pilot testing)

	TIME REQUIRED: One week

	STATUS: Not started


      - Update internals docs (Doxygen). Mostly this means updating
	function comments in the Python code, as the rest is
	automatic. May require a bit of overview text to explain the
	workings of the code, this overview text may well turn out to
	be just the current flat text documents marked up for
	inclusion by Doxygen.

	PRIORITY: Desirable

	TIME REQUIRED: One week.

	STATUS: Ongoing


      - Reorganize code (directory names, module names, which objects
	are in which modules, add gctx pointers to objects to avoid
	passing explicit gctx pointers in almost every function call)
	to make it easier to understand and maintain. Portions of the
	existing code were done in extreme haste to meet testing
	deadlines, and it shows.

	PRIORITY: Highly desirable

	TIME REQUIRED: One week.

	STATUS: Explicit gctx eradication done; much file renaming done; other
	stuff not started.


      - Add HSM support. Architecture includes it, current code does not. First
	step here would be talking to somebody with strong understanding of PKCS#
	11.

	PRIORITY: Desirable, not required for pilot

	TIME REQUIRED: Unknown

	STATUS: Not started


      - Installation packaging, so that rpkid can be built and
	installed like a normal package.

	PRIORITY: Desirable

	TIME REQUIRED: One week, longer if installation for many
	platforms is required

	STATUS: Not started


      - Tighten up syntax checking in left-right schema.

	PRIORITY: Desirable

	TIME REQUIRED: One day.

	STATUS: Not started


      - Rethink exposing SQL primary indices in protocols. Right now,
	auto-incremented SQL indices are used in many places in the
	left-right protocol, and are even exposed in a few places in
	our implementation of the up-down protocol. This is nice and
	unique but may be operationally fragile, since up-down usage
	means that URLs contain mechanically assigned identifiers
	rather than an identifier negotiated between the two parties
	during contract setup.

	The RIPE NCC suggested that we should instead use something
	like a hash of the client's name, which would be
	probabilistically unique, would not expose information, but
	would be stable even if we had to rebuild the database.

	PRIORITY: Rethinking desirable; reworking unknown

	TIME REQUIRED: One week to evaluate. Implementation time if we
	decide to make a change unknown, but probably on the order of
	another week.

	STATUS: Not started


      - Common protocol dump format with APNIC and other implementors so we can
	exchange protocol dumps.

	PRIORITY: Desirable

	TIME REQUIRED: Two days

	STATUS: Not started


      - IETF SIDR WG is still talking about ROAs with multiple
	signatures. No obvious need for this but IETF may mandate it
	anyway. Full implementation would require significant work
	revising current SQL table relations and upgrading CMS
	support.

	PRIORITY: Minimal, IETF feeping creaturism

	TIME REQUIRED: Unknown

	STATUS: Not started


      - Deaddrop of incoming messages, for audit.  Absent a better
        theory, steal existing tech for this: preface with minimal RFC
        2822 header and drop it into a Maildir folder using built-in
        Python Maildir library code, at which point it becomes soebody
        else's problem.

	STATUS: Not started

	PRIORITY: Desirable, trivial to implement.

      - Investigate using EKU (RFC 3280 4.2.1.13) as an alternative to
        wiring in BPKI EE certs for left-right protocol.



Other random notes:

Being able to specify interaction with other servers (not running
under testbed) in a testbed.yaml might be useful for interop tests.
Kind of breaks testbed's fundamental model, though.  Replacing what
testbed thinks is a leaf with somebody else would be easy, so maybe we
could specify some way to hang a bunch of rpkids under an external
parent?  Hmm, data needed would look a lot like testpoke.yaml, maybe
we can reuse some of that language?

There's a three-way tradeoff lurking in the publication protocol,
manifest generation, and CRL generation:

1) Consistancy issues for relying parties (eg, don't want to withdraw
   something that's still listed in the manifest);

2) Efficiency issues for the RPKI engine (eg, generating a new
   manifest for each individual change during a batch run could be
   expensive, would prefer to batch up the changes into a single
   manifest run); and

3) Coherency issues for the RPKI engine (don't want to defer things
   that could result in loss of state if something bad happens).

Considerations (1) and (3) have to dominate, which may mean we take a
hit on (2).

Most of the explicit calls to sql_fetch*() are now encapsulated in
one-line methods.  The remaining ones are probably hints at minor bits
of abstraction still to be done.

Biz certs currently used by test scripts don't include SKI or AKI.  I
think this is because the test scripts use "openssl x509" rather than
"openssl ca" when generating these certs.  Not critical, and will
probably become completely irrelevant with all-singing all-dancing
post-Amsterdam biz cert scripts, but should not be a big problem to
fix either if it gets in the way again.