diff options
Diffstat (limited to 'doc/manual/06.RPKI.RP.rcynic.wiki')
| -rw-r--r-- | doc/manual/06.RPKI.RP.rcynic.wiki | 690 |
1 files changed, 690 insertions, 0 deletions
diff --git a/doc/manual/06.RPKI.RP.rcynic.wiki b/doc/manual/06.RPKI.RP.rcynic.wiki new file mode 100644 index 00000000..6dc30ebe --- /dev/null +++ b/doc/manual/06.RPKI.RP.rcynic.wiki @@ -0,0 +1,690 @@ += rcynic RPKI validator =
+
+[[TracNav(doc/RPKI/TOC)]]
+[[PageOutline]]
+
+`rcynic` is the core RPKI relying party tool, and is the code which
+performs the actual RPKI validation. Most of the other relying party
+tools just use `rcynic`'s output.
+
+The name is short for "cynical rsync", because `rcynic`'s task involves
+an interleaved process of `rsync` retrieval and RPKI validation.
+
+This code was developed on FreeBSD, and has been tested most heavily
+on FreeBSD versions 6-STABLE through 8-STABLE. It is also known to
+work on Ubuntu (12.04 LTS), Debian (Wheezy) and Mac OS X (Snow
+Leopard). In theory it should run on any reasonably POSIX-like
+system. As far as we know, `rcynic` does not use any seriously
+non-portable features, but neither have we done a POSIX reference
+manual lookup for every function call. Please report any portability
+problems.
+
+== Don't panic ==
+
+`rcynic` has a lot of options, but it attempts to choose reasonable
+defaults where possible. The installation process will create a basic
+working `rcynic` configuration for you and arrange for this to run
+hourly under cron. If all goes well, this should "just work".
+
+`rcynic` has the ability to do all of its work in a chroot jail. This
+used to be the default configuration, but integrating this properly
+with platform-specific packaging systems (FreeBSD ports, `apt-get` on
+Ubuntu and Debian, etc) proved impractical. You can still get this
+behavior if you need it, by
+[[wiki:doc/RPKI/Installation/FromSource|installing from source]]
+and using the `--enable-rcynic-jail` option to `./configure`.
+
+The default configuration set up by `make install` and the various
+packaging systems will run `rcynic` under `cron` using the `rcynic-cron`
+wrapper script. See the
+[[wiki:doc/RPKI/RP/RunningUnderCron|instructions for setting up your own cron jobs]]
+if you need something more complicated; also see the
+[[wiki:doc/RPKI/RP/RunningUnderCron|instructions for setting up hierarchical rsync]]
+if you need to build a complex topology of rcynic validators.
+
+== Overview ==
+
+`rcynic` depends heavily on the OpenSSL `libcrypto` library, and
+requires a reasonably current version of OpenSSL with both RFC 3779
+and CMS support.
+
+`rcynic` expects all certificates, CRLs, and CMS objects to be in DER
+format. `rcynic` stores its database using filenames derived from the
+RPKI rsync URIs at which the data are published.
+
+All configuration is via an OpenSSL-style configuration file, except
+for selection of the name of the configuration file itself. A few
+other parameters can also be set from the command line. The default
+name for the configuration is "`rcynic.conf`"; you can override this
+with the `-c` option on the command line. The configuration file uses
+OpenSSL's configuration file syntax, and you can set OpenSSL library
+configuration paramaters (eg, "engine" settings) in the config file as
+well. `rcynic`'s own configuration parameters are in a section called
+"`[rcynic]`".
+
+Most configuration parameters are optional and have defaults which
+should do something reasonable if you are running `rcynic` in a test
+directory. If you're running rcynic as a system program, perhaps
+under `cron` via the `rcynic-cron` script, you'll want to set
+additional parameters to tell `rcynic` where to find its data and
+where to write its output (the installation process sets these
+parameters for you). The configuration file itself, however, is not
+optional. In order for `rcynic` to do anything useful, your
+configuration file **MUST** at minimum tell `rcynic` where to find one
+or more RPKI trust anchors or trust anchor locators (TALs).
+
+=== Trust anchors ===
+
+* To specify a trust anchor, use the `trust-anchor` directive to
+ name the local file containing the trust anchor.
+
+* To specify a trust anchor locator (TAL), use the
+ `trust-anchor-locator` directive to name a local file containing
+ the trust anchor locator.
+
+* To specify a directory containing trust anchors or trust anchor
+ locators, use the `trust-anchor-directory` directive to name the
+ directory. Files in the specified directory with names ending in
+ `".cer"` will be processed as trust anchors, while files with names
+ ending in `".tal"` will be processed as trust anchor locators.
+
+You may use a combination of these methods if necessary.
+
+Trust anchors are represented as DER-formatted X.509 self-signed
+certificate objects, but in practice trust anchor locators are more
+common, as they reduce the amount of locally configured data to the
+bare minimum and allow the trust anchor itself to be updated without
+requiring reconfiguration of validators like rcynic. A trust anchor
+locator is a file in the format specified in
+[[http://www.rfc-editor.org/rfc/rfc6490.txt|RFC-6490]], consisting of
+the rsync URI of the trust anchor followed by the Base64 encoding of
+the trust anchor's public key.
+
+Strictly speaking, trust anchors do not need to be self-signed, but
+many programs (including OpenSSL) assume that trust anchors will be
+self-signed. See the `allow-non-self-signed-trust-anchor`
+configuration option if you need to use a non-self-signed trust
+anchor, but be warned that the results, while technically correct, may
+not be useful.
+
+See the `make-tal.sh` script in this directory if you need to generate
+your own TAL file for a trust anchor.
+
+As of this writing, there still is no single global trust anchor for
+the RPKI system, so you have to provide separate trust anchors for
+each Regional Internet Registry (RIR) which is publishing RPKI data.
+The installation process installs the ones it knows about.
+
+Example of a minimal config file specifying nothing but trust anchor
+locators:
+
+{{{
+#!ini
+[rcynic]
+
+trust-anchor-locator.0 = trust-anchors/apnic.tal
+trust-anchor-locator.1 = trust-anchors/ripe.tal
+trust-anchor-locator.2 = trust-anchors/afrinic.tal
+trust-anchor-locator.3 = trust-anchors/lacnic.tal
+}}}
+
+Eventually, this should all be collapsed into a single trust anchor,
+so that relying parties don't need to sort this out on their own, at
+which point the above configuration could become something like:
+
+{{{
+#!ini
+[rcynic]
+
+trust-anchor-locator = trust-anchors/iana.tal
+}}}
+
+=== Output directories ===
+
+By default, `rcynic` uses two writable directory trees:
+
+`unauthenticated`::
+
+ Raw data fetched via `rsync`. In order to take full advantage
+ of `rsync`'s optimized transfers, you should preserve and reuse
+ this directory across `rcynic` runs, so that `rcynic` need not
+ re-fetch data that have not changed.
+
+`authenticated`::
+
+ Data which `rcynic` has checked. This is the real output of
+ the validation process.
+
+`authenticated` is really a symbolic link to a directory with a name of
+the form "`authenticated`.//<timestamp>//", where //<timestamp>// is an
+ISO 8601 timestamp like `2001-04-01T01:23:45Z`. `rcynic` creates a new
+timestamped directory every time it runs, and moves the symbolic link
+as an atomic operation when the validation process completes. The
+intent is that `authenticated` always points to the most recent usable
+validation results, so that programs which use `rcynic`'s output don't
+need to worry about whether an `rcynic` run is in progress.
+
+`rcynic` installs trust anchors specified via the `trust-anchor-locator`
+directive in the `unauthenticated` tree just like any other fetched
+object, and copies them into the `authenticated` trees just like any
+other object once they pass `rcynic`'s checks.
+
+`rcynic` copies trust anchors specified via the `trust-anchor` directive
+into the top level directory of the `authenticated` tree with filenames
+of the form //<xxxxxxxx>//`.`//<n>//`.cer`, where //<xxxxxxxx>// and
+//<n>// are the OpenSSL object name hash and index within the
+resulting virtual hash bucket, respectively. These are the same
+values that OpenSSL's `c_hash` Perl script would produce. The reason
+for this naming scheme is that these trust anchors, by definition, are
+not fetched automatically, and thus do not really have publication
+URIs in the sense that every other object in these trees do. So
+`rcynic` uses a naming scheme which insures:
+
+* that each trust anchor has a unique name within the output tree and
+
+* that trust anchors cannot be confused with certificates: trust
+ anchors always go in the top level of the tree, data fetched via
+ rsync always go in subdirectories.
+
+Trust anchors and trust anchor locators taken from the directory named
+by the `trust-anchor-directory` directive will follow the same naming
+scheme trust anchors and trust anchor locators specified via the
+`trust-anchor` and `trust-anchor-locator` directives, respectively.
+
+== Usage and configuration ==
+
+=== Logging levels ===
+
+`rcynic` has its own system of logging levels, similar to what
+`syslog()` uses, but customized to the specific task `rcynic` performs.
+
+||`log_sys_err` ||Error from operating system or library ||
+||`log_usage_err` ||Bad usage (local configuration error) ||
+||`log_data_err` ||Bad data (broken certificates or CRLs) ||
+||`log_telemetry` ||Normal chatter about rcynic's progress ||
+||`log_verbose` ||Extra verbose chatter ||
+||`log_debug` ||Only useful when debugging ||
+
+=== Command line options ===
+
+||`-c` //configfile// ||Path to configuration file (default: `rcynic.conf`) ||
+||`-l` //loglevel// ||Logging level (default: `log_data_err`) ||
+||`-s` ||Log via syslog ||
+||`-e` ||Log via stderr when also using syslog ||
+||`-j` ||Start-up jitter interval (see below; default: `600`) ||
+||`-V` ||Print rcynic's version to standard output and exit ||
+||`-x` ||Path to XML "summary" file (see below; no default) ||
+
+== Configuration file reference ==
+
+`rcynic` uses the OpenSSL `libcrypto` configuration file mechanism.
+All `libcrypto` configuration options (eg, for engine support) are
+available. All `rcynic`-specific options are in the "`[rcynic]`" section.
+You **MUST** have a configuration file in order for `rcynic` to do
+anything useful, as the configuration file is the only way to list
+your trust anchors.
+
+=== authenticated ===
+
+Path to output directory (where `rcynic` should place objects it
+has been able to validate).
+
+Default: `rcynic-data/authenticated`
+
+=== unauthenticated ===
+
+Path to directory where `rcynic` should store unauthenticatd
+data retrieved via `rsync`. Unless something goes horribly
+wrong, you want `rcynic` to preserve and reuse this directory
+across runs to minimize the network traffic necessary to bring
+your repository mirror up to date.
+
+Default: `rcynic-data/unauthenticated`
+
+=== rsync-timeout ===
+
+How long (in seconds) to let `rsync` run before terminating the
+`rsync` process, or zero for no timeout. You want this timeout
+to be fairly long, to avoid terminating `rsync` connections
+prematurely. It's present to let you defend against evil
+`rsync` server operators who try to tarpit your connection as a
+form of denial of service attack on `rcynic`.
+
+Default: `300`
+
+=== max-parallel-fetches ===
+
+Upper limit on the number of copies of `rsync` that `rcynic` is
+allowed to run at once. Used properly, this can speed up
+synchronization considerably when fetching from repositories
+built with sub-optimal tree layouts or when dealing with
+unreachable repositories. Used improperly, this option can
+generate excessive load on repositories, cause synchronization
+to be interrupted by firewalls, and generally creates create a
+public nuisance. Use with caution.
+
+As of this writing, values in the range 2-4 are reasonably
+safe. Values above 10 have been known to cause problems.
+
+`rcynic` can't really detect all of the possible problems
+created by excessive values of this parameter, but if rcynic's
+report shows that both successful retrivial and skipped
+retrieval from the same repository host, that's a pretty good
+hint that something is wrong, and an excessive value here is a
+good first guess as to the cause.
+
+Default: `1`
+
+=== rsync-program ===
+
+Path to the rsync program.
+
+Default: `rsync`, but you should probably set this variable rather
+than just trusting the `PATH` environment variable to be set
+correctly.
+
+=== log-level ===
+
+Same as `-l` option on command line. Command line setting overrides
+config file setting.
+
+Default: `log_log_err`
+
+=== use-syslog ===
+
+Same as `-s` option on command line. Command line setting overrides
+config file setting.
+
+Values: `true` or `false`.
+
+Default: `false`
+
+=== use-stderr ===
+
+Same as -e option on command line. Command line setting overrides
+config file setting.
+
+Values: `true` or `false`.
+
+Default: `false`, but if neither `use-syslog` nor `use-stderr` is set,
+log output goes to `stderr`.
+
+=== syslog-facility ===
+
+Syslog facility to use.
+
+Default: local0
+
+=== syslog-priority-xyz ===
+
+(where xyz is an rcynic logging level, above)
+
+Override the syslog priority value to use when
+logging messages at this rcynic level.
+
+Defaults:
+
+||`syslog-priority-log_sys_err` ||`err` ||
+||`syslog-priority-log_usage_err` ||`err` ||
+||`syslog-priority-log_data_err` ||`notice` ||
+||`syslog-priority-log_telemetry` ||`info` ||
+||`syslog-priority-log_verbose` ||`info` ||
+||`syslog-priority-log_debug` ||`debug` ||
+
+=== jitter ===
+
+Startup jitter interval, same as `-j` option on command line. Jitter
+interval, specified in number of seconds. `rcynic` will pick a random
+number within the interval from zero to this value, and will delay for
+that many seconds on startup. The purpose of this is to spread the
+load from large numbers of `rcynic` clients all running under cron
+with synchronized clocks, in particular to avoid hammering the global
+RPKI `rsync` servers into the ground at midnight UTC.
+
+Default: `600`
+
+=== lockfile ===
+
+Name of lockfile, or empty for no lock. If you run `rcynic` directly
+under cron, you should use this parameter to set a lockfile so that
+successive instances of rcynic don't stomp on each other. If you run
+`rcynic` under `rcynic-cron`, you don't need to touch this, as
+`rcynic-cron` maintains its own lock.
+
+Default: no lock
+
+=== xml-summary ===
+
+Enable output of a per-host summary at the end of an `rcynic`
+run in XML format.
+
+Value: filename to which XML summary should be written; "-" will send
+XML summary to standard output.
+
+Default: no XML summary.
+
+=== allow-stale-crl ===
+
+Allow use of CRLs which are past their `nextUpdate` timestamp.
+This is usually harmless, but since there are attack scenarios
+in which this is the first warning of trouble, it's
+configurable.
+
+Values: `true` or `false`.
+
+Default: `true`
+
+=== prune ===
+
+Clean up old files corresponding to URIs that `rcynic` did not
+see at all during this run. `rcynic` invokes `rsync` with the
+`--delete` option to clean up old objects from collections
+that `rcynic` revisits, but if a URI changes so that `rcynic`
+never visits the old collection again, old files will remain
+in the local mirror indefinitely unless you enable this
+option.
+
+Note: Pruning only happens when `run-rsync` is true. When the
+`run-rsync` option is false, pruning is not done regardless of
+the setting of the prune option option.
+
+Values: `true` or `false`.
+
+Default: `true`
+
+=== allow-stale-manifest ===
+
+Allow use of manifests which are past their `nextUpdate`
+timestamp. This is probably harmless, but since it may be an
+early warning of problems, it's configurable.
+
+Values: `true` or `false`.
+
+Default: `true`
+
+=== require-crl-in-manifest ===
+
+Reject publication point if manifest doesn't list the CRL that
+covers the manifest EE certificate.
+
+Values: `true` or `false`.
+
+Default: `false`
+
+=== allow-object-not-in-manifest ===
+
+Allow use of otherwise valid objects which are not listed in the
+manifest. This is not supposed to happen, but is probably harmless.
+
+Enabling this does, however, often result in noisier logs, as it
+increases the chance that `rcynic` will attempt to validate data which a
+CA removed from the manifest but did not completely remove and revoke
+from the repository.
+
+Values: `true` or `false`
+
+Default: `false`
+
+=== allow-digest-mismatch ===
+
+Allow use of otherwise valid objects which are listed in the
+manifest with a different digest value.
+
+You probably don't want to touch this.
+
+Values: `true` or `false`
+
+Default: `true`
+
+=== allow-crl-digest-mismatch ===
+
+Allow processing to continue on a publication point whose
+manifest lists a different digest value for the CRL than the
+digest of the CRL we have in hand.
+
+You probably don't want to touch this.
+
+Values: `true` or `false`
+
+Default: `true`
+
+=== allow-non-self-signed-trust-anchor ===
+
+Experimental. Attempts to work around OpenSSL's strong
+preference for self-signed trust anchors.
+
+We're not going to explain this one in any further detail. If you
+really want to know what it does, Use The Source, Luke.
+
+**Do not even consider enabling this option unless you are intimately
+familiar with both X.509 and the internals of OpenSSL's
+`X509_verify_cert()` function and really know what you are doing.**
+
+Values: `true` or `false`.
+
+Default: `false`
+
+=== run-rsync ===
+
+Whether to run `rsync` to fetch data. You don't generally want to
+change this except when building complex topologies where `rcynic`
+running on one set of machines acts as aggregators for another set of
+validators. A large ISP might want to build such a topology so that
+they could have a local validation cache in each POP while minimizing
+load on the global repository system and maintaining some degree of
+internal consistency between POPs. In such cases, one might want the
+`rcynic` instances in the POPs to validate data fetched from the
+aggregators via an external process, without the POP `rcynic`
+instances attempting to fetch anything themselves.
+
+Values: `true` or `false`.
+
+Default: `true`
+
+=== use-links ===
+
+Whether to use hard links rather than copying valid objects
+from the unauthenticated to authenticated tree. Using links
+is slightly more fragile (anything that stomps on the
+unauthenticated file also stomps on the authenticated file)
+but is a bit faster and reduces the number of inodes consumed
+by a large data collection. At the moment, copying is the
+default behavior, but this may change in the future.
+
+Values: `true` or `false`.
+
+Default: `false`
+
+=== rsync-early ===
+
+Whether to force `rsync` to run even when we have a valid manifest for
+a particular publication point and its `nextUpdate` time has not yet
+passed.
+
+This is an experimental feature, and currently defaults to **true**,
+which is the old behavior (running `rsync` regardless of whether we
+have a valid cached manifest). This default may change once we have
+more experience with `rcynic`'s behavior when run with this option set
+to `false`.
+
+Skipping the `rsync` fetch when we already have a valid cached
+manifest can significantly reduce the total number of `rsync`
+connections we need to make, and significantly reduce the load that
+each validator places on the authoritative publication servers. As
+with any caching scheme, however, there are some potential problems
+involved with not fetching the latest data, and we don't yet have
+enough experience with this option to know how this will play out in
+practice, which is why this is still considered experimental.
+
+Values: `true` or `false`
+
+Default: `true` (but may change in the future)
+
+=== trust-anchor ===
+
+Specify one RPKI trust anchor, represented as a local file
+containing an X.509 certificate in DER format. Value of this
+option is the pathname of the file.
+
+**No default**.
+
+=== trust-anchor-locator ===
+
+Specify one RPKI trust anchor locator, represented as a local file in
+the format specified in
+[[http://www.rfc-editor.org/rfc/rfc6490.txt|RFC-6490]]. This a simple
+text format containing an rsync URI and the RSA public key of the
+X.509 object specified by the URI; the first line of the file is the
+URI, the remainder is the public key in Base64 encoded DER format.
+
+Value of this option is the pathname of the file.
+
+**No default**.
+
+=== trust-anchor-directory ===
+
+Specify a directory containing trust anchors, trust anchor locators,
+or both. Trust anchors in such a directory must have filenames ending
+in "`.cer`"; trust anchor locators in such a directory must have names
+ending in "`.tal`"; any other files will be skipped.
+
+This directive is an alternative to using the `trust-anchor` and
+trust-anchor-locator` directives. This is probably easier to use than
+the other trust anchor directives when dealing with a collection of
+trust anchors. This may change on that promised day when we have only
+a single global trust anchor to deal with, but we're not there yet.
+
+**No default**.
+
+== Post-processing rcynic's XML output ==
+
+The distribution includes several post-processors for the XML output
+`rcynic` writes describing the actions it has taken and the validation
+status of the objects it has found.
+
+=== rcynic-html === #rcynichtml
+
+`rcynic-html` converts `rcynic`'s XML output into a collection of HTML
+pages summarizing the results, noting problems encountered, and
+showing some history of `rsync` transfer times and repository object
+counts in graphical form.
+
+`rcynic-cron` runs `rcynic-html` automatically, immediately after running
+`rcynic`. If for some reason you need to run `rcynic-html` by hand, the
+command syntax is:
+
+{{{
+#!sh
+$ rcynic-html rcynic.xml /web/server/directory/
+}}}
+
+`rcynic-html` will write a collection of HTML and image files to the
+specified output directory, along with a set of RRD databases.
+`rcynic-html` will create the output directory if necessary.
+
+`rcynic-html` requires [[http://www.rrdtool.org/|`rrdtool`]], a
+specialized database and graphing engine designed for this sort of
+work. You can run `rcynic-html` without `rrdtool` by giving it the
+`--no-show-graphs` option, but the result won't be as useful.
+
+`rcynic-html` gets its idea of where to find the `rrdtool` program from
+autoconf, which usually works. If for some reason it doesn't work in
+your environment, you will need to tell `rcynic-html` where to find
+`rrdtool`, using the `--rrdtool-binary` option:
+
+{{{
+#!sh
+$ rcynic-html --rrdtoolbinary /some/where/rrdtool rcynic.xml /web/server/directory/
+}}}
+
+=== rcynic.xsl ===
+
+`rcynic.xsl` was an earlier attempt at the same kind of HTML output as
+[[#rcynichtml|rcynic-html]] generates. XSLT was a convenient language
+for our initial attempts at this, but as the processing involved got
+more complex, it became obvious that we needed a general purpose
+programming language.
+
+If for some reason XSLT works better in your environment than Python,
+you might find this stylesheet to be a useful starting point, but be
+warned that it's significantly slower than `rcynic-html`, lacks many
+features, and is no longer under development.
+
+=== rcynic-text ===
+
+`rcynic-text` provides a quick flat text summary of validation results.
+This is useful primarily in test scripts
+([[wiki:doc/RPKI/CA#smoketest|smoketest]] uses it).
+
+Usage:
+
+{{{
+#!sh
+$ rcynic-text rcynic.xml
+}}}
+
+=== validation_status ===
+
+`validation_status` provides a flat text translation of the detailed
+validation results. This is useful primarily for checking the
+detailed status of some particular object or set of objects, perhaps
+using a program like `grep` or `awk` to filter `validation_status`'s
+output.
+
+Usage:
+
+{{{
+#!sh
+$ validation_status rcynic.xml
+$ validation_status rcynic.xml | fgrep rpki.misbehaving.org
+$ validation_status rcynic.xml | fgrep object_rejected
+}}}
+
+=== rcynic-svn ===
+
+`rcynic-svn` is a tool for archiving `rcynic`'s results in a
+[[http://subversion.apache.org/|Subversion]] repository. `rcynic-svn`
+is not integrated into `rcynic-cron`, because this is not something
+that every relying party is going to want to do. However, for relying
+parties who want to analyze `rcynic`'s output over a long period of
+time, `rcynic-svn` may provide a useful starting point starting point.
+
+To use `rcynic-svn`, you first must set up a Subversion repository and
+check out a working directory:
+
+{{{
+#!sh
+$ svnadmin create /some/where/safe/rpki-archive
+$ svn co file:///some/where/safe/rpki-archive /some/where/else/rpki-archive
+}}}
+
+The name can be anything you like, in this example we call it
+"`rpki-archive`". The above sequence creates the repository, then
+checks out an empty working directory `/some/where/else/rpki-archive`.
+
+The repository does not need to be on the same machine as the working
+directory, but it probably should be for simplicity unless you have
+some strong need to put it elsewhere.
+
+Once you have the repository and working directory set up, you need to
+arrange for `rcynic-svn` to be run after each `rcynic` run whose results
+you want to archive. One way to do this would be to run `rcynic-svn` in
+the same cron job as `rcynic-cron`, immediately after `rcynic-cron` and
+specifying the same lock file that `rcynic-cron` uses.
+
+Sample usage, assuming that `rcynic`'s data is in the usual place:
+
+{{{
+#!sh
+$ rcynic-svn --lockfile /var/rcynic/data/lock \
+ /var/rcynic/data/authenticated \
+ /var/rcynic/data/unauthenticated \
+ /var/rcynic/data/rcynic.xml \
+ /some/where/else/rpki-archive
+}}}
+
+where the last argument is the name of the Subversion working
+directory and the other arguments are the names of those portions of
+`rcynic`'s output which you wish to archive. Generally, the above set
+(`authenticated`, `unauthenticated`, and `rcynic.xml`) are the ones
+you want, but feel free to experiment.
|