First cut at documentation on the publication protocol

svn path=/docs/publication-protocol; revision=1862

1 files changed, 244 insertions, 167 deletions

diff --git a/docs/publication-protocol b/docs/publication-protocol
index 1346776a..466a4cf1 100644
--- a/docs/publication-protocol
+++ b/docs/publication-protocol
@@ -1,167 +1,244 @@
-;;; -*- Lisp -*-
-;;; $Id$
-
-;;; Copyright (C) 2007--2008  American Registry for Internet Numbers ("ARIN")
-;;;
-;;; Permission to use, copy, modify, and distribute this software for any
-;;; purpose with or without fee is hereby granted, provided that the above
-;;; copyright notice and this permission notice appear in all copies.
-;;;
-;;; THE SOFTWARE IS PROVIDED "AS IS" AND ARIN DISCLAIMS ALL WARRANTIES WITH
-;;; REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
-;;; AND FITNESS.  IN NO EVENT SHALL ARIN BE LIABLE FOR ANY SPECIAL, DIRECT,
-;;; INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
-;;; LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
-;;; OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
-;;; PERFORMANCE OF THIS SOFTWARE.
-
-;;; Scratch pad for working out API design for RPKI engine.
-;;;
-;;; This file is psuedocode, I just wanted to take advantage of
-;;; emacs's built-in support for languages with reasonable syntax.
-;;;
-;;; Terminology:
-;;;
-;;; - IRBE: Internet Registry Back End
-;;;
-;;; - RE: RPKI Engine
-
-;;; Repository update protocol.  Same basic CMS-signed XML mess we use
-;;; elsewhere, this time with RE as client, lodging repository as
-;;; server.  Authorization is a combination of business key and
-;;; resource key/cert: biz key demonstrates that we're authorized to
-;;; play with this repository at all, resource cert demonstrates
-;;; relationship to the datum to be published.
-
-;;; This would be a trivial protocol, except for two constraints:
-;;;
-;;; a) In the case where parent and child are sharing a repository,
-;;;    we'd like to nest child under parent to speed up rcynic.
-;;;
-;;; b) The repository operator might want to do some checks to assure
-;;;    itself that what it's about to allow the RE to publish is not
-;;;    dangerous toxic waste.
-;;;
-;;; Protocol goals as of last discussion in Prague:
-;;;
-;;; 1) "Negotiate" publication point URI.  This is a bit complex, in
-;;;     that part of the URI may come from the publication
-;;;     repository, parts from the RE's parent, and parts from the RE
-;;;     itself.  Some discussion over whether we need online protocol
-;;;     to negotiate the publication repository's part, no firm
-;;;     consensus, plurality of the room was leaning towards "no",
-;;;     ie, the repository's portion of the URI is negotiated via the
-;;;     business channel and configured into the RE by the IRBE.
-;;;
-;;; 2) In case where parent and child are in fact sharing repository,
-;;;    repository operator is responsible for keeping these two users'
-;;;    naming schemes from coliding with each other.  More precisely,
-;;;    the repository operator is responsible for checking to see that
-;;;    the parent agreed to let the child publish in a particular
-;;;    subtree.  Fortunately, this is an easy check: the parent has to
-;;;    issue a cert to the child anyway, and that cert will contain a
-;;;    signature by the parent over the child's SIA URI, so the
-;;;    repository just has to check that.  For purposes of this check,
-;;;    the parent's cert (which the repository has, by definition,
-;;;    since this is the nested hosting case) can serve as a trust
-;;;    anchor for checking the child's SIA.
-;;;
-;;; 3) To the extent that the repository operator wants to guard
-;;;    against toxic waste, it might want to check further up the
-;;;    resource cert chain, regardless of where the ancestors lodge.
-;;;    For this to work properly, the repository operator needs to
-;;;    agree (as part of a business negotiation, probably) with the RE
-;;;    on which trust anchors the repository should use to perform
-;;;    these checks; ultimately, this decision (as with any TA choice)
-;;;    is up to the relying party (in this case the repository
-;;;    operator), but if there's going to be a problem due to
-;;;    mismatched TA choices, we would really like to throw the
-;;;    exception during the business negotiation rather than via a
-;;;    runtime refusal to publish.
-;;;
-;;; Note that, in this publication model, any agreement that the
-;;; repository makes to publish the RE's output is conditional upon
-;;; the object to be published passing all of its checks.
-
-;;; How do we construct publication URIs (which also go into some of
-;;; the X.509 extensions in the resource certs)?  We create CAs on the
-;;; fly in response to what we learn from our parent, so it's hard to
-;;; preconfigure this.  This mechanism is still under discussion, the
-;;; following is my version of it.
-;;;
-;;; At least for purposes of discussion, break the publication
-;;; directory URI into three pieces: head/middle/tail/.
-;;;
-;;; head is a URI within the repository with which this RE publishes;
-;;; this is either per-parent or per-class-per-parent, but the latter
-;;; is hard to preconfigure because we only find out about classes on
-;;; the fly.  So, for the moment, assume it's per-parent.  We're only
-;;; allowed to publish stuff here because we have a business
-;;; relationship with the repository, so at some level this has to be
-;;; preconfigured anyway, along with the repository TA and contact
-;;; URI.  In theory we could negotiate a location within the
-;;; repository on the fly, but let's try to keep this simple.
-;;;
-;;; Middle may come from this RE's parent.  If the parent happens to
-;;; be using the same repository as this RE is, the parent can tell us
-;;; (currently via an attribute I added to the up-down protocol for
-;;; this purpose) a URI under which it gives us permission to lodge.
-;;; If the head URI (configured above) is not a prefix of the URI we
-;;; get from the parent, we don't have permission to publish under the
-;;; parent and middle is null.  In essence, middle is the parent's
-;;; advice on where to put this particular CA's outputs in order to
-;;; get the nice hierarchical properties we want.
-;;;
-;;; Tail is something this RE makes up.  It's per-CA, and all that
-;;; really matters is that it's stable.  It could be gensymed, or
-;;; could be our internal name for the CA, whatever.
-;;;
-;;; Publication itself always requires a business signature
-;;; (demonstrating that we have the right to publish in this
-;;; repository at all) and may also require enough of the RPKI cert
-;;; chain to demonstrate that this RE's parent has given this RE
-;;; permission to publish under a particular URI.  Thing that needs to
-;;; be proven is that publication client A is not stepping on
-;;; publication client B even when B is A's parent.
-
-;;; Signed manifests must be supplied by the RE, as they must be
-;;; signed by an EE cert issued by the CA that issues (or signs, in
-;;; the case of non-cert objects) everything else in the SIA
-;;; collection.  This EE cert should probably just use the RFC 3779
-;;; inherit bits as an easy way of inheriting all the resources of the
-;;; CA cert.  So the publication operation supplies exactly one new
-;;; manifest and zero or more other objects; everything in the
-;;; publication PDU is published as an atomic operation, ie, if any of
-;;; it can't be published, none of it is published.
-
-(publish-thing :publication-uri uri-of-thing-we-are-publishing
-	       :credential-certs (cert ....)
-	       :manifest manifest
-	       (:thing crl1 :thing-type :crl)
-	       (:thing cert1 :thing-type :cert)
-	       (:thing cert2 :thing-type :cert)
-	       ...	       ...)
-=> ()
-
-;;; thing is an object (certificate, CRL, ROA) signed by the private
-;;; key associated with a resource certificate held by the entity
-;;; sending the (publish-thing) request.  :thing-type may not be
-;;; strictly necessary.
-;;;
-;;; credential-certs is a set of whatever resource certificates are
-;;; needed to demonstrate to the repository engine that the entity
-;;; requesting publication is making a legitimate publication request.
-;;; Goal (2), above, requires the requestor to supply the resource
-;;; certificate chain up to the parent to demonstrate that the parent
-;;; has approved (signed) the requested SIA.  Goal (3), above, would
-;;; require supplying the cert chain back to some resource trust
-;;; anchor established as part of the business relationship between
-;;; requestor and repository operator.
-
-;;; Hmm, the above completely ignores deletion.  Geoff's docs say that
-;;; when a resource class disappears we're supposed to withdraw all
-;;; certs, CRLs, and manifests associated with that resource class
-;;; from the publication point.  Strictly speaking, this isn't
-;;; necessary, since the parent has presumably CRLed the subtree in
-;;; question, but it'd be polite to relying parties, to avoid wasting
-;;; their time.
+-*- Text -*-
+$Id$
+
+Copyright (C) 2007--2008  American Registry for Internet Numbers ("ARIN")
+
+Permission to use, copy, modify, and distribute this software for any
+purpose with or without fee is hereby granted, provided that the above
+copyright notice and this permission notice appear in all copies.
+
+THE SOFTWARE IS PROVIDED "AS IS" AND ARIN DISCLAIMS ALL WARRANTIES WITH
+REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
+AND FITNESS.  IN NO EVENT SHALL ARIN BE LIABLE FOR ANY SPECIAL, DIRECT,
+INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
+LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
+OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
+PERFORMANCE OF THIS SOFTWARE.
+
+@section Terminology
+
+@li @em IRBE: Internet Registry Back End
+
+@li @em IRDB: Internet Registry Data Base
+
+@li @em BPKI: Business PKI
+
+@li @em RPKI: Resource PKI
+
+@section Protocol operations between IRBE and RPKI engine
+
+The publication protocol is really two separate client/server
+protocols, between different parties.  The first is a configuration
+protocol for the IRBE to use to configure the publication engine, the
+second is the interface by which authorized clients request
+publication of specific objects.
+
+Much of the architecture of the publication protocol is borrowed from
+the left-right protocol: like the left-right protocol, the publication
+protocol uses CMS-wrapped XML over HTTPS with the same eContentType
+OID and the same HTTPS content-type, and the overall style of the XML
+messages is very similar to the left-right protocol.  All operations
+allow an optional "tag" attribute to allow batching.
+
+The publication engine operates a single HTTPS server which serves
+both of these subprotocols.  The two subprotocols share a single
+server port, but use distinct URLs.
+
+@subsection Publication control subprotocol
+
+The control subprotocol reuses the message-passing design of the
+left-right protocol.  Configured objects support the "create", "set",
+"get", "list", and "destroy" actions, or a subset thereof when the
+full set of actions doesn't make sense.
+
+@subsubsection <config/> object
+
+The <config/> object allows configuration of data that apply to the
+entire publication server rather than a particular client.
+
+There is exactly one <config/> object in the publication server, and
+it only supports the "set" and "get" actions -- it cannot be created
+or destroyed.
+
+Payload data which can be configured in a <config/> object:
+
+@li @c bpki_crl (element):
+
+    This is the BPKI CRL used by the publication server when signing
+    the CMS wrapper on responses in the publication subprotocol.  As
+    the CRL must be updated at regular intervals, it's not practical
+    to restart the publication server when the BPKI CRL needs to be
+    updated.  Fortunately, the BPKI model doesn't require use of a
+    BPKI CRL between the IRBE and the publication server, so we can
+    use the publication control subprotocol to update the BPKI CRL.
+
+@subsubsection <client/> object
+
+The <client/> object represents one client authorized to use the
+publication server.
+
+The <client/> object supports the full set of "create", "set", "get",
+"list", and "destroy" actions.  Each client has a "client_id"
+attribute, which is used in responses and must be specified in "set",
+"get", or "destroy" actions.
+
+Payload data which can be configured in a <client/> object:
+
+@li @c base_uri (attribute):
+
+    This is the base URI below which this client is allowed to publish
+    data.  The publication server may impose additional constraints in
+    the case of a child publishing beneath its parent.
+
+@li bpki_cert (element):
+
+    BPKI CA certificate for this <client/>.  This is used as part of
+    the certificate chain when validating incoming TLS and CMS
+    messages.  If the bpki_glue certificate is in use (below), the
+    bpki_cert certificate should be issued by the bpki_glue
+    certificate; otherwise, the bpki_cert certificate should be issued
+    by the publication engine's bpki_ta certificate.
+
+@li bpki_glue (element):
+
+    Another BPKI CA certificate for this <client/>, usually not
+    needed.  Certain pathological cross-certification cases require a
+    two-certificate chain due to issuer name conflicts.  If used, the
+    bpki_glue certificate should be the issuer of the bpki_cert
+    certificate and should be issued by the publication engine's
+    bpki_ta certificate; if not needed, the bpki_glue certificate
+    should be left unset.
+
+@subsection Publication subprotocol
+
+The publication subprotocol is structured somewhat differently from
+the publication control protocol.  Objects in the publication
+subprotocol represent objects to be published or objects to be
+withdrawn from publication.  Each kind of object supports two actions:
+"publish" and "withdraw".  In each case the XML element representing
+hte object to be published or withdrawn has a "uri" attribute which
+contains the publication URI.  For "publish" actions, the XML element
+body contains the DER object to be published, encoded in Base64; for
+"withdraw" actions, the XML element body is empty.
+
+In theory, the detailed access control for each kind of object might
+be different.  In practice, as of this writing, access control for all
+objects is a simple check that the client's @c "base_uri" is a leading
+substring of the publication URI.  Details of why access control might
+need to become more complicated are discussed in a later section.
+
+@subsubsection <certificate/> object
+
+The <certificate/> object represents an RPKI certificate to be
+published or withdrawn.
+
+@subsubsection <crl/> object
+
+The <crl/> object represents an RPKI CRL to be published or withdrawn.
+
+@subsubsection <manifest/> object
+
+The <manifest/> object represents an RPKI publication manifest to be
+published or withdrawn.
+
+Note that part of the reason for the batching support in the
+publication protocol is because @em every publication or withdrawal
+action requires a new manifest, thus every publication or withdrawal
+action will involve at least two objects.
+
+@subsubsection <roa/> object
+
+The <roa/> object represents a ROA to be published or withdrawn.
+
+@subsection Error handling
+
+Error in this protocol are handled at two levels.
+
+Since all messages in this protocol are conveyed over HTTPS
+connections, basic errors are indicated via the HTTP response code.
+4xx and 5xx responses indicate that something bad happened.  Errors
+that make it impossible to decode a query or encode a response are
+handled in this way.
+
+Where possible, errors will result in a <report_error/> message which
+takes the place of the expected protocol response message.
+<report_error/> messages are CMS-signed XML messages like the rest of
+this protocol, and thus can be archived to provide an audit trail.
+
+<report_error/> messages only appear in replies, never in queries.
+The <report_error/> message can appear on either the "forward" (IRBE
+as client of RPKI engine) or "back" (RPKI engine as client of IRDB)
+communication channel.
+
+The <report_error/> message includes an optional @c "tag" attribute to
+assist in matching the error with a particular query when using
+batching, and also includes a @c "self_id" attribute indicating the
+<self/> that issued the error.
+
+The error itself is conveyed in the @c error_code (attribute).  The
+value of this attribute is a token indicating the specific error that
+occurred.  At present this will be the name of a Python exception; the
+production version of this protocol will nail down the allowed error
+tokens here, probably in the RelaxNG schema.
+
+The body of the <report_error/> element itself is an optional text
+string; if present, this is debugging information.  At present this
+capabilty is not used, debugging information goes to syslog.
+
+@section Additional access control considerations.
+
+As detailed above, the publication protocol is trivially simple.  This
+glosses over two bits of potential complexity:
+
+@li In the case where parent and child are sharing a repository, we'd
+    like to nest child under parent, because testing has demonstrated
+    that even on relatively slow hardware the delays involved in
+    setting up separate rsync connections tend to dominate
+    synchronization time for relying parties.
+
+@li The repository operator might also want to do some checks to
+    assure itself that what it's about to allow the RPKI engine to
+    publish is not dangerous toxic waste.
+
+The up-down protocol includes a mechanism by which a parent can
+suggest a publication URI to each of its children.  The children are
+not required to accept this hint, and the children must make separate
+arrangements with the repository operator (who might or might not be
+the same as the entity that hosts the children's RPKI engine
+operations) to use the suggested publication point, but if everything
+works out, this allows children to nest cleanly under their parents
+publication points, which helps reduce synchronization time for
+relying parties.
+
+In this case, one could argue that the publication server is
+responsible for preventing one of its clients (the child in the above
+description) from stomping on data published by another of its clients
+(the parent in the above description).  This goes beyond the basic
+access check and requires the publication server to determine whether
+the parent has given its consent for the child to publish under the
+parent.  Since the RPKI certificate profile requires the child's
+publication point to be indicated in an SIA extension in a certificate
+issued by the parent to the child, the publication engine can infer
+this permission from the parent's issuance of a certificate to the
+child.  Since, by definition, the parent also uses this publication
+server, this is an easy check, as the publication server should
+already have the parent's certificate available by the time it needs
+to check the child's certificate.
+
+The previous paragraph only covers a "publish" action for a
+<certificate/> object.  For "publish" actions on other objects, the
+publication server would need to trace permission back to the
+certificate issued by the parent; for "withdraw" actions, the
+publication server would have to perform the same checks it would
+perform for a "publish" action, using the current published data
+before withdrawing it.  The latter in turn implies an ordering
+constraint on "withdraw" actions in order to preserve the data
+necessary for these access control decisions; as this may prove
+impractical, the publication server may probably need to make periodic
+sweeps over its published data looking for orphaned objects, but
+that's probably a good idea anyway.
+
+Note that, in this publication model, any agreement that the
+repository makes to publish the RPKI engine's output is conditional
+upon the object to be published passing whatever access control checks
+the publication server imposes.