path: root/openssl/README

$Id$ -*- Text -*- 

OpenSSL hacked to add support for the RFC 3779 X.509 v3 extensions.

Current status:

- Not (yet) for distribution outside the RIRs.

- Reads and writes RFC 3779 extensions but does not (yet) perform the
  additional validation described in RFC 3779 2.3 & 3.3.

- Not (yet) tested extensively.

Please report any problems to me (sra) or the rescert mailing list.



This is what the current openssl.conf syntax looks like for the RFC
3779 certificate extensions.  Syntax is admittedly wretched, because
it has to work with the existing OpenSSL code.  Within that
restriction, I've attempted to make this look as much as practical
like the existing OpenSSL support for "multi-valued" extensions.  RFC
3779 ASN.1 provided for easy reference.

Notes:

* Ranges are denoted with a hyphen, prefix lengths with a slash.
  I could tag ranges differently from the atomic types, but this
  seemed easier for the user to understand.

* The "@" syntax indicating indirection through a separate section
  is lifted from the stock OpenSSL multi-valued extension support.

* I didn't attempt to guess which addresses are IPv4 and which are
  IPv6 from the syntax, since the opensssl.conf multi-value syntax
  needs tags anyway.

* SAFI support is present but minimal.  If you want a SAFI, you have
  to specify its numeric value.  It would be trivial to add additional
  keywords for specific SAFIs if there were a reason to do so.

* The "sbgp-" names were already present in OpenSSL's table of known
  extension OIDs.  We can talk to the folks at the OpenSSL project
  about changing the names if there's a reason to do so.

### 

# An address extension, all specified on one line

sbgp-ipAddrBlock = critical, IPv4:10.1.1.1/32, IPv4:10.2.0.0-10.3.255.255

# An address extension, all specified on one line, with inheritance

sbgp-ipAddrBlock = critical, IPv4:inherit, IPv6:2002::/16

# An address extension using SAFIs

sbgp-ipAddrBlock = critical, IPv4-SAFI:1:10.1.1.1/32, IPv6-SAFI:1:2002::/16

# Address extension using an indirect section

sbgp-ipAddrBlock = critical, @addr-section

[addr-section]

IPv4.0 = 10.0.0.1
IPv4.1 = 10.0.1.0/24
IPv4.2 = 10.2.0.0 - 10.3.255.255
IPv6.0 = 2002:1::/64
IPv6.1 = 2002:2:: - 2002:8::ffff:ffff:ffff:ffff:ffff

###

# An ASID extension, all specified on one line:

sbgp-autonomousSysNum = critical, AS:44, RDI:33-45

# ASID extension on one line using inheritance

sbgp-autonomousSysNum = critical, AS:55, RDI:inherit

# ASID extension using an indirect section

sbgp-autonomousSysNum = critical, @asid-section

[asid-section]

AS.0  = 44
AS.1  = 55 - 77
RDI.0 = 33



Some notes on OpenSSL internals.

O'Reilly "Network Security with OpenSSL" is a bit dated (four years
old, corresponds roughly to OpenSSL 0.9.7), but still appears to be
the closest thing there is to coherent documentation.  Some updates
and machine readable copies of examples are available at
http://www.opensslbook.com/.  In spite of its age, the book is useful
as it gives a readable overview of bit and pieces of OpenSSL's
internal programming environment which one would otherwise have to
absorb from the code via osmosis.  Chapter 10 is particularly useful,
as are the sections on error handling and abstract I/O (the ERR and
BIO packages, respectively) in chapter 4.

OpenSSL's own doc is very patchy, although fairly extensive in places.
Most of it eventually comes down to "Use the Source, Luke" with
pointers on which bit of source serves as an interesting example.

For x509v3 extentions, the place to start is doc/openssl.txt,
which, oddly, turns out to be mostly about certificate extensions.  It
gives an overview of the mechanisms, in particular of the method
routine interface for certificate extensions.  For something like the
RFC 3779 extensions, it's pretty clear that we need to use a "raw"
extension (which, as far as I can tell, just means that the RFC 3779
stuff is complicated enough that the extension handler has to do a lot
of work to deal with a complex ASN.1 structure that the rest of the
code doesn't know much about).

General note on global symbols in OpenSSL: always look at the header
file for any global symbol you're using.  In fact, it's probably best
to do a global search (m-x tags-search if you're an emacs user) for
all instances of a global symbol before attempting to use it, as there
are a lot of things that one just has to know about how all the global
stuff hangs together.  There are header files full of magic
definitions that one just has to one need to be extended.  There are
magic pre-sorted lists of handlers that one just needs to know about.
Little or none of this is documented.  Use the Source, Luke.

In some cases -portions- of files are automatically generated by Perl
scripts (eg, the per-type stack definitions in safestack.h).  Ouch.
If you see a large block of very repetitive stuff, check for comments
indicating that it's automatically generated.

Oh, and the indentation style is demented.

Header files you definitely need to read if you're going to touch this
stuff:

    crypto/x509v3/x509v3.h
    crypto/x509v3/ext_dat.h
    crypto/stack/safestack.h
    crypto/asn1/asn1t.h
    crypto/asn1/asn1.h
    crypto/objects/objects.h

Automatically generated header files you'll need to skim, then go read
the input files listed in the header comments and perhaps the
generating Perl code:

    crypto/objects/obj_mac.h
    crypto/objects/obj_dat.h

Much of the code shows a heavy Perl influence, presumably dating back
clear to Eric Young.  Some of the internal data structure operators
have names that only make sense to a Perl programmer.  Stacks are
really lists, and may be sorted.  Where code is automatically
generated, it's done by Perl scripts.  The configuration language for
the whole package is a Perl script.  Assembly code is all wrapped up
inside perl scripts in a moderately clever attempt at being able to
write the assembly language only once and use it with various
assemblers with nontrivially different syntax.  Much of the
documentation markup (including manual pages) for the C code is .pod.

I have not yet figured out where to hook in the extra goop that RFC
3779 will need for verification.  Making extensions critical is easy
enough, but the validation stuff in RFC 3779 2.3 and 3.3 needs to go
somewhere.

A lot of the missing documentation is buried in ssleay.txt, which the
other documentation says not to read because it's so old.  But it's
where Eric explains all the basic data structures and expected usage
as of the dawn of time, so most of the stuff that's so old that it's
undocumented is really documented there.

xxx_new() functions set pointers of sub-structures to NULL or allocate
the substructures (one can leak memory if one doesn't know or check
which a particular xxx_new() function has done...), and the xxx_free()
functions clean up complex structures.  So be sure to set unused
pointers to NULL if one has been fiddling.

Make sure that memory leak detection (CRYPTO_MDEBUG) is turned on when
debugging.

"make update" in the top level runs all the magic perl code that
grovels through the code generating error codes, safestacks, etc.
util/mkstack.pl finds DECLARE_STACK_OF() declarations and generates
safestack definitions automatically if you run "make update".  Be
afraid.  Be very very afraid.

My initial test configuration was:

   ./Configure debug -DDEBUG_SAFESTACK

which tried to pull in -lefence (/usr/ports/devel/ElectricFence), so I
installed that.  Sadly, ElectricFence is not kidding when it says it
is very very very slow, but it was the bignum debugging printouts that
were driving me nuts, so I ended up creating my own "debug-sra"
configuration for the options I want.



Random reminders and notes to myself:

- The new stuff in crypto/x509v3/x509v3.h is not organized properly
  yet, it's all lumped in one place rather than being interleved with
  the other supported extensions.  This was to make it easier for me
  to debug, but will proably need to be cleaned up eventually.

- The reference for RDIs in RFC 3779 is incorrect.  I've been told
  that the authoritative definition of RDIs is ISO 10747, available as
  http://www.acm.org/sigs/sigcomm/standards/iso_stds/IDRP/10747.TXT.

- "openssl verify" only accepts PEM, not DER, which is annoying.
  An -inform switch would be nice, but the library routines
  don't know how to read a CAfile full of DER anyway.  Pity -CApath
  doesn't seem to work for us.  Oh well, live with PEM for now.

- OpenSSL already checks for duplicate extensions: more precisely,
  unless we explicitly tell X509_get_ext_d2i() that we allow multiple
  extensions (by providing the idx parameter), it returns NULL if it
  finds duplicates.  If we really want to check for presence of
  exactly one extension of a particular type, we call this function
  twice with the idx parameter and make sure that the second call
  returns NULL.

- May need to check AKID in crypto/x509/x509_vfy.c:get_crl().

- Request extensions: see X509_REQ_get_extensions(),
  X509_REQ_add_extensions(), and copy_extensions().



The June meeting at APNIC came up with a list of desired OpenSSL
primitives (transcribed by Geoff, thanks!).  Current status, with
notes and questions at the end.

1. Read a resource certificate and print nominated certificate
   field(s) (CLI command) or access data structure elements from the
   certificate (API)

   Status: Done

   CLI: openssl x509 -text

   API: i2r_ASIdentifiers(), i2r_IPAddrBlocks(), but in practice you'd
   rarely call these directly -- they're hooked into OpenSSL's
   X509V3_EXT_METHOD dispatch mechanism and cached by
   x509v3_cache_extensions(), so you'd call X509_get_ext_d2i() or just
   look for decoded data structures hanging off the X509 structure.

2. Verify a resource certificate (use the existing openssl interface
   which provides as inputs a) a set of 'trusted' certs and CRLs and
   b) a set of untrusted certs and CRLs and c) the cert to be
   verified. output is YES or NO and both a CLI and an API interface
   is desired

   Status: Done, for some definition of done (CLI program remains a
   crock, but I don't think I've made it worse than it already was).

   CLI: openssl verify

   API: Pre-existing X509_verify_cert() function now checks RFC 3779
   path validation, which in turn checks for well formed extensions.

3. 3779_canonicalise - read in a data structure of a resource set and
   produce as output the 3779 canonical form of the resource set - the
   CLI interface will print this to stdout and the API interface will
   pass a ref to a data structure (or fill in a data struct or
   whatever makes sense!)

   Status: Done.

   CLI: openssl x509, openssl req, ....   Called automatically as part
   of anything that reads RFC 3779 extensions from openssl.conf.

   API: v3_asid_canonize(), v3_addr_canonize().

4. 3779_cmp reads in 2 x 3779_canonicalised data sets and outputs a
   comparison = EQUAL is the two are equal, or =SUBSET if data1 is a
   strict subset of data2, or = NOT in all other cases (CLI or API)
   (EQUAL, SUBSET, NOT)

   Status: API code written, not fully tested.  No CLI.  API functions
   test whether an extension uses inheritance, and whether one
   extension is a (possibly improper) subset of another.  Subset test
   fails if either extension uses inheritance.

   API: v3_asid_inherits(), v3_addr_inherits(), v3_asid_subset(),
   v3_addr_subset().

5. is_3379_canonical tests a single data set and returns CANONICAL if
   the resource is formatted according to 3779 or NOT is otherwise

   Status: Done.

   CLI: openssl verify calls this during path validation.  No separate
   program to perform just this function, would not be hard to write
   one if it were needed.

   API: v3_asid_is_canonical(), v3_addr_is_canonical().

6. is_in_cert takes a certificate and a resource set description and
   checks if the certificate 'covers' the resource set The outpouts
   are EQUAL if the resource cert precisely matches the resource set,
   SUBSET if the resource set is a subset of the certificate resource
   list, or NOT otherwise

   Status: Done.

   API: v3_asid_validate_resource_set(), v3_addr_validate_resource_set().
   These return true if a certificate chain covers a resource set.
   "Resource sets" are represented as the C form of the appropriate
   extension, with the additional constraint that the resource set
   must not use inheritance; this constraint is enforced by the code
   (ie, using one of these functions on a resource set that specifies
   inheritance will always return false regardless of the contents of
   the chain).

   CLI: resource-set-test.  Use the Source, Luke.

7. generate_resource_certificate generates a resource certificate -
   I'm not sure I understand what the inputs are to be here - perhaps
   a data structure of the fields and values, but this should be
   confirmed. the output is a DER object (or pem, or either
   selectable?)

   Status: Done, for some definition of done.

   CLI: openssl x509, openssl ca.

   API: Preexisting OpenSSL functions, augmented by method routines
   allowing OpenSSL to read and write RFC 3779 extensions.

8. generate a certificate request (previous note on which certificate
   request format to use is on the table)

   Status: Done, for some definition of done, for OpenSSL's native
   request format (PKCS10 DER, with or without Base64 "PEM" wrapper).

   CLI: openssl x509, openssl req.

   API: Preexisting OpenSSL functions, augmented by method routines
   allowing OpenSSL to read and write RFC 3779 extensions.

9. process a certificate request and say yes / no on whether the
   request is well formed.

   Status: Request processing within OpenSSL looks to be a mess, with
   entirely too much of it happening in applications rather than the
   library.  Other API functions exist to test whether an extension is
   in canonical form, etcetera.  Path validation can only be done in
   context of a particular certificate chain, which is probably out of
   scope for this primative.

10. process a certificate request and generate a certificate in
    response.

   Status: OpenSSL code is a mess, see (9).  Generating certificate
   based on a certificate request is prexisting functionality; new
   code adds ability to process RFC 3779.  If task here includes path
   validation with respect to signer's certificate chain, would
   require nontrivial changes to the OpenSSL CLI programs that sign
   requests, as none of them currently even look for certificate
   chains for the signer.

   API: Primitive #6 (above) extended to accept argument indicating
   whether inheritance is allowed, so that primitive #6 can be used to
   perform path validation on extensions pulled from a request.

Notes:

- "For some definition of done": opinions vary on whether the existing
  OpenSSL CLI tools are adaquate for the purposes for which people use
  them.  For purposes of the above discussion, I'm assuming that they
  are, and that my task was just to teach the same tools to deal with
  the RFC 3779 extensions.

- It turns out that the code I've written so far never needs to ask
  whether two extension data sets are equal, just whether one is a
  subset (possibly improper) of the other.   Checking for equality is
  a somewhat different test, so rather than trying to combine the two,
  I might just provide separate functions that test for equality.



Final code cleanups before submitting patch to OpenSSL project.

- Ben requested compile-time conditionals and Configure support for
  them.

  - Code changes: done.

  - Configure support: do we need anything beyond -DOPENSSL_RFC3779 ?
    Need to read existing Configure code to see if there's a preferred
    way of doing this.

- License adjustments per discusion with ARIN and OpenSSL
  project.  Done.

- Some of the static constructor functions ought to be global so that
  users of the library can call them.  In some cases we'd want trivial
  jacket functions calling existing code, eg, all the asid stuff that
  takes a pointer to an ASIdentifierChoice should probably take a
  pointer to the ASIdentifier and an integer indicating AS vs RDI.
  This of course begs the question of whether the internal calls
  should be rewritten to use the new external calls, ie, there's some
  potential cleanup work here.

  Candidates: constructors:

  - asid_add_id_or_range()	[minor rewrite, done]

  - asid_add_inherit()		[minor rewrite, done]

  - addr_add_range()            [rename, done]

  - addr_add_prefix()           [rename, done]

  - addr_add_inherit()          [rename, done]

  Candidates: manipulation of obscure ASN.1 encodings (v3_addr.c)

  - afi_from_addressfamily()    [renamed, now public]

  - extract_min_max()		[v3_addr_get_range()]

  - addr_expand()		[v3_addr_get_range()]

  - addr_prefixlen()		[v3_addr_get_range()]

  - length_from_afi()           [v3_addr_get_range()]

  v3_addr_get_range() is a new public function that encapsulates
  several of the private functions, by hiding some details and
  wrapping a lot of error checking around extract_min_max().

- Need patch against OpenSSL HEAD as well as 0.9.8 branch.  Figure out
  how to do that, or ask Ben for help.