Preliminary version of rpkid et al with all the TLS code ripped out.

Not quite ready for cutover yet, may need some conversion tools and
instructions, but checking this into a branch (well, sort of) so that
others can look at the code changes involved, try it out themselves,
etc.  At some point this will merge back into rpkid/ directory and
there will be only one, without TLS, but converting the testbed is
going to require a flag day, so need to keep the TLS version around
until then.

svn path=/rpkid.without_tls; revision=3449

1 files changed, 468 insertions, 0 deletions

diff --git a/rpkid.without_tls/doc/Left-Right b/rpkid.without_tls/doc/Left-Right
new file mode 100644
index 00000000..54502d7e
--- /dev/null
+++ b/rpkid.without_tls/doc/Left-Right
@@ -0,0 +1,468 @@
+****** Left-Right Protocol ******
+
+The left-right protocol is really two separate client/server protocols over
+separate channels between the RPKI engine and the IR back end (IRBE).
+
+The IRBE is the client for one of the subprotocols, the RPKI engine is the
+client for the other.
+
+***** initiated by the IRBE *****
+
+This part of the protcol uses a kind of message-passing. Each object that the
+RPKI engine knows about takes five messages: "create", "set", "get", "list",
+and "destroy". Actions which are not just data operations on objects are
+handled via an SNMP-like mechanism, as if they were fields to be set. For
+example, to generate a keypair one "sets" the "generate-keypair" field of a BSC
+object, even though there is no such field in the object itself as stored in
+SQL. This is a bit of a kludge, but the reason for doing it as if these were
+variables being set is to allow composite operations such as creating a BSC,
+populating all of its data fields, and generating a keypair, all as a single
+operation. With this model, that's trivial, otherwise it's at least two round
+trips.
+
+Fields can be set in either "create" or "set" operations, the difference just
+being whether the object already exists. A "get" operation returns all visible
+fields of the object. A "list" operation returns a list containing what "get"
+would have returned on each of those objects.
+
+Left-right protocol objects are encoded as signed CMS messages containing XML
+as eContent and using an eContentType OID of id-ct-xml
+(1.2.840.113549.1.9.16.1.28). These CMS messages are in turn passed as the data
+for HTTPS POST operations, with an HTTP content type of "application/x-rpki"
+for both the POST data and the response data.
+
+All operations allow an optional "tag" attribute which can be any alphanumeric
+token. The main purpose of the tag attribute is to allow batching of multiple
+requests into a single PDU.
+
+**** <self/> object ****
+
+A <self/> object represents one virtual RPKI engine. In simple cases where the
+RPKI engine operator operates the engine only on their own behalf, there will
+only be one <self/> object, representing the engine operator's organization,
+but in environments where the engine operator hosts other entities, there will
+be one <self/> object per hosted entity (probably including the engine
+operator's own organization, considered as a hosted customer of itself).
+
+Some of the RPKI engine's configured parameters and data are shared by all
+hosted entities, but most are tied to a specific <self/> object. Data which are
+shared by all hosted entities are referred to as "per-engine" data, data which
+are specific to a particular <self/> object are "per-self" data.
+
+Since all other RPKI engine objects refer to a <self/> object via a
+"self_handle" value, one must create a <self/> object before one can usefully
+configure any other left-right protocol objects.
+
+Every <self/> object has a self_handle attribute, which must be specified for
+the "create", "set", "get", and "destroy" actions.
+
+Payload data which can be configured in a <self/> object:
+
+  use_hsm (attribute):
+      Whether to use a Hardware Signing Module. At present this option has no
+      effect, as the implementation does not yet support HSMs.
+
+  crl_interval (attribute):
+      Positive integer representing the planned lifetime of an RPKI CRL for
+      this <self/>, measured in seconds.
+
+  regen_margin (attribute):
+      Positive integer representing how long before expiration of an RPKI
+      certificiate a new one should be generated, measured in seconds. At
+      present this only affects the one-off EE certificates associated with
+      ROAs. This parameter also controls how long before the nextUpdate time of
+      CRL or manifest the CRL or manifest should be updated.
+
+  bpki_cert (element):
+      BPKI CA certificate for this <self/>. This is used as part of the
+      certificate chain when validating incoming TLS and CMS messages, and
+      should be the issuer of cross-certification BPKI certificates used in
+      <repository/>, <parent/>, and <child/> objects. If the bpki_glue
+      certificate is in use (below), the bpki_cert certificate should be issued
+      by the bpki_glue certificate; otherwise, the bpki_cert certificate should
+      be issued by the per-engine bpki_ta certificate.
+
+  bpki_glue (element):
+      Another BPKI CA certificate for this <self/>, usually not needed. Certain
+      pathological cross-certification cases require a two-certificate chain
+      due to issuer name conflicts. If used, the bpki_glue certificate should
+      be the issuer of the bpki_cert certificate and should be issued by the
+      per-engine bpki_ta certificate; if not needed, the bpki_glue certificate
+      should be left unset.
+
+Control attributes that can be set to "yes" to force actions:
+
+  rekey:
+      Start a key rollover for every RPKI CA associated with every <parent/
+      > object associated with this <self/> object. This is the first phase of
+      a key rollover operation.
+
+  revoke:
+      Revoke any remaining certificates for any expired key associated with any
+      RPKI CA for any <parent/> object associated with this <self/> object.
+      This is the second (cleanup) phase for a key rollover operation; it's
+      separate from the first phase to leave time for new RPKI certificates to
+      propegate and be installed.
+
+  reissue:
+      Not implemented, may be removed from protocol. Original theory was that
+      this operation would force reissuance of any object with a changed key,
+      but as that happens automatically as part of the key rollover mechanism
+      this operation seems unnecessary.
+
+  run_now:
+      Force immediate processing for all tasks associated with this <self/
+      > object that would ordinarily be performed under cron. Not currently
+      implemented.
+
+  publish_world_now:
+      Force (re)publication of every publishable object for this <self/
+      > object. Not currently implemented. Intended to aid in recovery if RPKI
+      engine and publication engine somehow get out of sync.
+
+**** <bsc/> object ****
+
+The <bsc/> ("business signing context") object represents all the BPKI data
+needed to sign outgoing CMS or HTTPS messages. Various other objects include
+pointers to a <bsc/> object. Whether a particular <self/> uses only one <bsc/
+> or multiple is a configuration decision based on external requirements: the
+RPKI engine code doesn't care, it just cares that, for any object representing
+a relationship for which it must sign messages, there be a <bsc/> object that
+it can use to produce that signature.
+
+Every <bsc/> object has a bsc_handle, which must be specified for the "create",
+"get", "set", and "destroy" actions. Every <bsc/> also has a self_handle
+attribute which indicates the <self/> object with which this <bsc/> object is
+associated.
+
+Payload data which can be configured in a <isc/> object:
+
+  signing_cert (element):
+      BPKI certificate to use when generating a signature.
+
+  signing_cert_crl (element):
+      CRL which would list signing_cert if it had been revoked.
+
+Control attributes that can be set to "yes" to force actions:
+
+  generate_keypair:
+      Generate a new BPKI keypair and return a PKCS #10 certificate request.
+      The resulting certificate, once issued, should be configured as this
+      <bsc/> object's signing_cert.
+
+Additional attributes which may be specified when specifying
+"generate_keypair":
+
+  key_type:
+      Type of BPKI keypair to generate. "rsa" is both the default and, at the
+      moment, the only allowed value.
+
+  hash_alg:
+      Cryptographic hash algorithm to use with this keypair. "sha256" is both
+      the default and, at the moment, the only allowed value.
+
+  key_length:
+      Length in bits of the keypair to be generated. "2048" is both the default
+      and, at the moment, the only allowed value.
+
+Replies to "create" and "set" actions that specify "generate-keypair" include a
+<bsc_pkcs10/> element, as do replies to "get" and "list" actions for a <bsc/
+> object for which a "generate-keypair" command has been issued. The RPKI
+engine stores the PKCS #10 request, which allows the IRBE to reuse the request
+if and when it needs to reissue the corresponding BPKI signing certificate.
+
+**** <parent/> object ****
+
+The <parent/> object represents the RPKI engine's view of a particular parent
+of the current <self/> object in the up-down protocol. Due to the way that the
+resource hierarchy works, a given <self/> may obtain resources from multiple
+parents, but it will always have at least one; in the case of IANA or an RIR,
+the parent RPKI engine may be a trivial stub.
+
+Every <parent/> object has a parent_handle, which must be specified for the
+"create", "get", "set", and "destroy" actions. Every <parent/> also has a
+self_handle attribute which indicates the <self/> object with which this
+<parent/> object is associated, a bsc_handle attribute indicating the <bsc/
+> object to be used when signing messages sent to this parent, and a
+repository_handle indicating the <repository/> object to be used when
+publishing issued by the certificate issued by this parent.
+
+Payload data which can be configured in a <parent/> object:
+
+  peer_contact_uri (attribute):
+      HTTPS URI used to contact this parent.
+
+  sia_base (attribute):
+      The leading portion of an rsync URI that the RPKI engine should use when
+      composing the publication URI for objects issued by the RPKI certificate
+      issued by this parent.
+
+  sender_name (attribute):
+      Sender name to use in the up-down protocol when talking to this parent.
+      The RPKI engine doesn't really care what this value is, but other
+      implementations of the up-down protocol do care.
+
+  recipient_name (attribute):
+      Recipient name to use in the up-down protocol when talking to this
+      parent. The RPKI engine doesn't really care what this value is, but other
+      implementations of the up-down protocol do care.
+
+  bpki_cms_cert (element):
+      BPKI CMS CA certificate for this <parent/>. This is used as part of the
+      certificate chain when validating incoming CMS messages If the
+      bpki_cms_glue certificate is in use (below), the bpki_cms_cert
+      certificate should be issued by the bpki_cms_glue certificate; otherwise,
+      the bpki_cms_cert certificate should be issued by the bpki_cert
+      certificate in the <self/> object.
+
+  bpki_cms_glue (element):
+      Another BPKI CMS CA certificate for this <parent/>, usually not needed.
+      Certain pathological cross-certification cases require a two-certificate
+      chain due to issuer name conflicts. If used, the bpki_cms_glue
+      certificate should be the issuer of the bpki_cms_cert certificate and
+      should be issued by the bpki_cert certificate in the <self/> object; if
+      not needed, the bpki_cms_glue certificate should be left unset.
+
+  bpki_https_cert (element):
+      BPKI HTTPS CA certificate for this <parent/>. This is like the
+      bpki_cms_cert object, only used for validating incoming TLS messages
+      rather than CMS.
+
+  bpki_cms_glue (element):
+      Another BPKI HTTPS CA certificate for this <parent/>, usually not needed.
+      This is like the bpki_cms_glue certificate, only used for validating
+      incoming TLS messages rather than CMS.
+
+Control attributes that can be set to "yes" to force actions:
+
+  rekey:
+      This is like the rekey command in the <self/> object, but limited to RPKI
+      CAs under this parent.
+
+  reissue:
+      This is like the reissue command in the <self/> object, but limited to
+      RPKI CAs under this parent.
+
+  revoke:
+      This is like the revoke command in the <self/> object, but limited to
+      RPKI CAs under this parent.
+
+**** <child/> object ****
+
+The <child/> object represents the RPKI engine's view of particular child of
+the current <self/> in the up-down protocol.
+
+Every <child/> object has a child_handle, which must be specified for the
+"create", "get", "set", and "destroy" actions. Every <child/> also has a
+self_handle attribute which indicates the <self/> object with which this
+<child/> object is associated.
+
+Payload data which can be configured in a <child/> object:
+
+  bpki_cert (element):
+      BPKI CA certificate for this <child/>. This is used as part of the
+      certificate chain when validating incoming TLS and CMS messages. If the
+      bpki_glue certificate is in use (below), the bpki_cert certificate should
+      be issued by the bpki_glue certificate; otherwise, the bpki_cert
+      certificate should be issued by the bpki_cert certificate in the <self/
+      > object.
+
+  bpki_glue (element):
+      Another BPKI CA certificate for this <child/>, usually not needed.
+      Certain pathological cross-certification cases require a two-certificate
+      chain due to issuer name conflicts. If used, the bpki_glue certificate
+      should be the issuer of the bpki_cert certificate and should be issued by
+      the bpki_cert certificate in the <self/> object; if not needed, the
+      bpki_glue certificate should be left unset.
+
+Control attributes that can be set to "yes" to force actions:
+
+  reissue:
+      Not implemented, may be removed from protocol.
+
+**** <repository/> object ****
+
+The <repository/> object represents the RPKI engine's view of a particular
+publication repository used by the current <self/> object.
+
+Every <repository/> object has a repository_handle, which must be specified for
+the "create", "get", "set", and "destroy" actions. Every <repository/> also has
+a self_handle attribute which indicates the <self/> object with which this
+<repository/> object is associated.
+
+Payload data which can be configured in a <repository/> object:
+
+  peer_contact_uri (attribute):
+      HTTPS URI used to contact this repository.
+
+  bpki_cms_cert (element):
+      BPKI CMS CA certificate for this <repository/>. This is used as part of
+      the certificate chain when validating incoming CMS messages If the
+      bpki_cms_glue certificate is in use (below), the bpki_cms_cert
+      certificate should be issued by the bpki_cms_glue certificate; otherwise,
+      the bpki_cms_cert certificate should be issued by the bpki_cert
+      certificate in the <self/> object.
+
+  bpki_cms_glue (element):
+      Another BPKI CMS CA certificate for this <repository/>, usually not
+      needed. Certain pathological cross-certification cases require a two-
+      certificate chain due to issuer name conflicts. If used, the
+      bpki_cms_glue certificate should be the issuer of the bpki_cms_cert
+      certificate and should be issued by the bpki_cert certificate in the
+      <self/> object; if not needed, the bpki_cms_glue certificate should be
+      left unset.
+
+  bpki_https_cert (element):
+      BPKI HTTPS CA certificate for this <repository/>. This is like the
+      bpki_cms_cert object, only used for validating incoming TLS messages
+      rather than CMS.
+
+  bpki_cms_glue (element):
+      Another BPKI HTTPS CA certificate for this <repository/>, usually not
+      needed. This is like the bpki_cms_glue certificate, only used for
+      validating incoming TLS messages rather than CMS.
+
+At present there are no control attributes for <repository/> objects.
+
+**** <route_origin/> object ****
+
+This section is out-of-date. The <route_origin/> object has been replaced by
+the <list_roa_requests/> IRDB query, but the documentation for that hasn't been
+written yet.
+
+The <route_origin/> object is a kind of prototype for a ROA. It contains all
+the information needed to generate a ROA once the RPKI engine obtains the
+appropriate RPKI certificates from its parent(s).
+
+Note that a <route_origin/> object represents a ROA to be generated on behalf
+of <self/>, not on behalf of a <child/>. Thus, a hosted entity that has no
+children but which does need to generate ROAs would be represented by a hosted
+<self/> with no <child/> objects but one or more <route_origin/> objects. While
+lumping ROA generation in with the other RPKI engine activities may seem a
+little odd at first, it's a natural consequence of the design requirement that
+the RPKI daemon never transmit private keys across the network in any form;
+given this requirement, the RPKI engine that holds the private keys for an RPKI
+certificate must also be the engine which generates any ROAs that derive from
+that RPKI certificate.
+
+The precise content of the <route_origin/> has changed over time as the
+underlying ROA specification has changed. The current implementation as of this
+writing matches what we expect to see in draft-ietf-sidr-roa-format-03, once it
+is issued. In particular, note that the exactMatch boolean from the -02 draft
+has been replaced by the prefix and maxLength encoding used in the -03 draft.
+
+Payload data which can be configured in a <route_origin/> object:
+
+  asn (attribute):
+      Autonomous System Number (ASN) to place in the generated ROA. A single
+      ROA can only grant authorization to a single ASN; multiple ASNs require
+      multiple ROAs, thus multiple <route_origin/> objects.
+
+  ipv4 (attribute):
+      List of IPv4 prefix and maxLength values, see below for format.
+
+  ipv6 (attribute):
+      List of IPv6 prefix and maxLength values, see below for format.
+
+Control attributes that can be set to "yes" to force actions:
+
+  suppress_publication:
+      Not implemented, may be removed from protocol.
+
+The lists of IPv4 and IPv6 prefix and maxLength values are represented as
+comma-separated text strings, with no whitespace permitted. Each entry in such
+a string represents a single prefix/maxLength pair.
+
+ABNF for these address lists:
+
+     <ROAIPAddress> ::= <address> "/" <prefixlen> [ "-" <max_prefixlen> ]
+                           ; Where <max_prefixlen> defaults to the same
+                           ; value as <prefixlen>.
+
+     <ROAIPAddressList> ::= <ROAIPAddress> *( "," <ROAIPAddress> )
+
+For example, "10.0.1.0/24-32,10.0.2.0/24", which is a shorthand form of
+"10.0.1.0/24-32,10.0.2.0/24-24".
+
+***** Operations initiated by the RPKI engine *****
+
+The left-right protocol also includes queries from the RPKI engine back to the
+IRDB. These queries do not follow the message-passing pattern used in the IRBE-
+initiated part of the protocol. Instead, there's a single query back to the
+IRDB, with a corresponding response. The CMS and HTTPS encoding are the same as
+in the rest of the protocol, but the BPKI certificates will be different as the
+back-queries and responses form a separate communication channel.
+
+**** <list_resources/> messages ****
+
+The <list_resources/> query and response allow the RPKI engine to ask the IRDB
+for information about resources assigned to a particular child. The query must
+include both a "self_handle" attribute naming the <self/> that is making the
+request and also a "child_handle" attribute naming the child that is the
+subject of the query. The query and response also allow an optional "tag"
+attribute of the same form used elsewhere in this protocol, to allow batching.
+
+A <list_resources/> response includes the following attributes, along with the
+tag (if specified), self_handle, and child_handle copied from the request:
+
+  valid_until:
+      A timestamp indicating the date and time at which certificates generated
+      by the RPKI engine for these data should expire. The timestamp is
+      expressed as an XML xsd:dateTime, must be expressed in UTC, and must
+      carry the "Z" suffix indicating UTC.
+
+  asn:
+      A list of autonomous sequence numbers, expressed as a comma-separated
+      sequence of decimal integers with no whitespace.
+
+  ipv4:
+      A list of IPv4 address prefixes and ranges, expressed as a comma-
+      separated list of prefixes and ranges with no whitespace. See below for
+      format details.
+
+  ipv6:
+      A list of IPv6 address prefixes and ranges, expressed as a comma-
+      separated list of prefixes and ranges with no whitespace. See below for
+      format details.
+
+Entries in a list of address prefixes and ranges can be either prefixes, which
+are written in the usual address/prefixlen notation, or ranges, which are
+expressed as a pair of addresses denoting the beginning and end of the range,
+written in ascending order separated by a single "-" character. This format is
+superficially similar to the format used for prefix and maxLength values in the
+<route_origin/> object, but the semantics differ: note in particular that
+<route_origin/> objects don't allow ranges, while <list_resources/> messages
+don't allow a maxLength specification.
+
+***** Error handling *****
+
+Error in this protocol are handled at two levels.
+
+Since all messages in this protocol are conveyed over HTTPS connections, basic
+errors are indicated via the HTTP response code. 4xx and 5xx responses indicate
+that something bad happened. Errors that make it impossible to decode a query
+or encode a response are handled in this way.
+
+Where possible, errors will result in a <report_error/> message which takes the
+place of the expected protocol response message. <report_error/> messages are
+CMS-signed XML messages like the rest of this protocol, and thus can be
+archived to provide an audit trail.
+
+<report_error/> messages only appear in replies, never in queries. The
+<report_error/> message can appear on either the "forward" (IRBE as client of
+RPKI engine) or "back" (RPKI engine as client of IRDB) communication channel.
+
+The <report_error/> message includes an optional "tag" attribute to assist in
+matching the error with a particular query when using batching, and also
+includes a "self_handle" attribute indicating the <self/> that issued the
+error.
+
+The error itself is conveyed in the error_code (attribute). The value of this
+attribute is a token indicating the specific error that occurred. At present
+this will be the name of a Python exception; the production version of this
+protocol will nail down the allowed error tokens here, probably in the RelaxNG
+schema.
+
+The body of the <report_error/> element itself is an optional text string; if
+present, this is debugging information. At present this capabilty is not used,
+debugging information goes to syslog.