path: root/doc/doc.RPKI.RP.rcynic

****** rcynic RPKI validator ******

rcynic is the core RPKI relying party tool, and is the code which performs the
actual RPKI validation. Most of the other relying party tools just use rcynic's
output.

The name is short for "cynical rsync", because rcynic's task involves an
interleaved process of rsync retrieval and RPKI validation.

This code was developed on FreeBSD, and has been tested most heavily on FreeBSD
versions 6-STABLE through 8-STABLE. It is also known to work on Ubuntu (12.04
LTS), Debian (Wheezy) and Mac OS X (Snow Leopard). In theory it should run on
any reasonably POSIX-like system. As far as we know, rcynic does not use any
seriously non-portable features, but neither have we done a POSIX reference
manual lookup for every function call. Please report any portability problems.

***** Don't panic *****

rcynic has a lot of options, but it attempts to choose reasonable defaults
where possible. The installation process will create a basic working rcynic
configuration for you and arrange for this to run hourly under cron. If all
goes well, this should "just work".

rcynic has the ability to do all of its work in a chroot jail. This used to be
the default configuration, but integrating this properly with platform-specific
packaging systems (FreeBSD ports, apt-get on Ubuntu and Debian, etc) proved
impractical. You can still get this behavior if you need it, by installing from
source and using the --enable-rcynic-jail option to ./configure.

The default configuration set up by make install and the various packaging
systems will run rcynic under cron using the rcynic-cron wrapper script. See
the instructions for setting up your own cron jobs if you need something more
complicated; also see the instructions for setting up hierarchical rsync if you
need to build a complex topology of rcynic validators.

***** Overview *****

rcynic depends heavily on the OpenSSL libcrypto library, and requires a
reasonably current version of OpenSSL with both RFC 3779 and CMS support.

rcynic expects all certificates, CRLs, and CMS objects to be in DER format.
rcynic stores its database using filenames derived from the RPKI rsync URIs at
which the data are published.

All configuration is via an OpenSSL-style configuration file, except for
selection of the name of the configuration file itself. A few other parameters
can also be set from the command line. The default name for the configuration
is "rcynic.conf"; you can override this with the -c option on the command line.
The configuration file uses OpenSSL's configuration file syntax, and you can
set OpenSSL library configuration paramaters (eg, "engine" settings) in the
config file as well. rcynic's own configuration parameters are in a section
called "[rcynic]".

Most configuration parameters are optional and have defaults which should do
something reasonable if you are running rcynic in a test directory. If you're
running rcynic as a system program, perhaps under cron via the rcynic-cron
script, you'll want to set additional parameters to tell rcynic where to find
its data and where to write its output (the installation process sets these
parameters for you). The configuration file itself, however, is not optional.
In order for rcynic to do anything useful, your configuration file MUST at
minimum tell rcynic where to find one or more RPKI trust anchors or trust
anchor locators (TALs).

**** Trust anchors ****

* To specify a trust anchor, use the trust-anchor directive to name the local
  file containing the trust anchor.

* To specify a trust anchor locator (TAL), use the trust-anchor-locator
  directive to name a local file containing the trust anchor locator.

* To specify a directory containing trust anchors or trust anchor locators, use
  the trust-anchor-directory directive to name the directory. Files in the
  specified directory with names ending in ".cer" will be processed as trust
  anchors, while files with names ending in ".tal" will be processed as trust
  anchor locators.

You may use a combination of these methods if necessary.

Trust anchors are represented as DER-formatted X.509 self-signed certificate
objects, but in practice trust anchor locators are more common, as they reduce
the amount of locally configured data to the bare minimum and allow the trust
anchor itself to be updated without requiring reconfiguration of validators
like rcynic. A trust anchor locator is a file in the format specified in RFC-
6490, consisting of the rsync URI of the trust anchor followed by the Base64
encoding of the trust anchor's public key.

Strictly speaking, trust anchors do not need to be self-signed, but many
programs (including OpenSSL) assume that trust anchors will be self-signed. See
the allow-non-self-signed-trust-anchor configuration option if you need to use
a non-self-signed trust anchor, but be warned that the results, while
technically correct, may not be useful.

See the make-tal.sh script in this directory if you need to generate your own
TAL file for a trust anchor.

As of this writing, there still is no single global trust anchor for the RPKI
system, so you have to provide separate trust anchors for each Regional
Internet Registry (RIR) which is publishing RPKI data. The installation process
installs the ones it knows about.

Example of a minimal config file specifying nothing but trust anchor locators:

  [rcynic]

  trust-anchor-locator.0 = trust-anchors/apnic.tal
  trust-anchor-locator.1 = trust-anchors/ripe.tal
  trust-anchor-locator.2 = trust-anchors/afrinic.tal
  trust-anchor-locator.3 = trust-anchors/lacnic.tal

Eventually, this should all be collapsed into a single trust anchor, so that
relying parties don't need to sort this out on their own, at which point the
above configuration could become something like:

  [rcynic]

  trust-anchor-locator = trust-anchors/iana.tal

**** Output directories ****

By default, rcynic uses two writable directory trees:

unauthenticated::

     Raw data fetched via rsync. In order to take full advantage of
     rsync's optimized transfers, you should preserve and reuse this
     directory across rcynic runs, so that rcynic need not re-fetch data
     that have not changed.

authenticated::

     Data which rcynic has checked. This is the real output of the
     validation process.

authenticated is really a symbolic link to a directory with a name of the form
"authenticated.<timestamp>", where <timestamp> is an ISO 8601 timestamp like
2001-04-01T01:23:45Z. rcynic creates a new timestamped directory every time it
runs, and moves the symbolic link as an atomic operation when the validation
process completes. The intent is that authenticated always points to the most
recent usable validation results, so that programs which use rcynic's output
don't need to worry about whether an rcynic run is in progress.

rcynic installs trust anchors specified via the trust-anchor-locator directive
in the unauthenticated tree just like any other fetched object, and copies them
into the authenticated trees just like any other object once they pass rcynic's
checks.

rcynic copies trust anchors specified via the trust-anchor directive into the
top level directory of the authenticated tree with filenames of the form
<xxxxxxxx>.<n>.cer, where <xxxxxxxx> and <n> are the OpenSSL object name hash
and index within the resulting virtual hash bucket, respectively. These are the
same values that OpenSSL's c_hash Perl script would produce. The reason for
this naming scheme is that these trust anchors, by definition, are not fetched
automatically, and thus do not really have publication URIs in the sense that
every other object in these trees do. So rcynic uses a naming scheme which
insures:

* that each trust anchor has a unique name within the output tree and

* that trust anchors cannot be confused with certificates: trust anchors always
  go in the top level of the tree, data fetched via rsync always go in
  subdirectories.

Trust anchors and trust anchor locators taken from the directory named by the
trust-anchor-directory directive will follow the same naming scheme trust
anchors and trust anchor locators specified via the trust-anchor and trust-
anchor-locator directives, respectively.

***** Usage and configuration *****

**** Logging levels ****

rcynic has its own system of logging levels, similar to what syslog() uses, but
customized to the specific task rcynic performs.

log_sys_err   Error from operating system or library

log_usage_err Bad usage (local configuration error)

log_data_err  Bad data (broken certificates or CRLs)

log_telemetry Normal chatter about rcynic's progress

log_verbose   Extra verbose chatter

log_debug     Only useful when debugging

**** Command line options ****

-c configfile Path to configuration file (default: rcynic.conf)

-l loglevel   Logging level (default: log_data_err)

-s            Log via syslog

-e            Log via stderr when also using syslog

-j            Start-up jitter interval (see below; default: 600)

-V            Print rcynic's version to standard output and exit

-x            Path to XML "summary" file (see below; no default)

***** Configuration file reference *****

rcynic uses the OpenSSL libcrypto configuration file mechanism. All libcrypto
configuration options (eg, for engine support) are available. All rcynic-
specific options are in the "[rcynic]" section. You MUST have a configuration
file in order for rcynic to do anything useful, as the configuration file is
the only way to list your trust anchors.

**** authenticated ****

Path to output directory (where rcynic should place objects it has been able to
validate).

Default: rcynic-data/authenticated

**** unauthenticated ****

Path to directory where rcynic should store unauthenticatd data retrieved via
rsync. Unless something goes horribly wrong, you want rcynic to preserve and
reuse this directory across runs to minimize the network traffic necessary to
bring your repository mirror up to date.

Default: rcynic-data/unauthenticated

**** rsync-timeout ****

How long (in seconds) to let rsync run before terminating the rsync process, or
zero for no timeout. You want this timeout to be fairly long, to avoid
terminating rsync connections prematurely. It's present to let you defend
against evil rsync server operators who try to tarpit your connection as a form
of denial of service attack on rcynic.

Default: 300

**** max-parallel-fetches ****

Upper limit on the number of copies of rsync that rcynic is allowed to run at
once. Used properly, this can speed up synchronization considerably when
fetching from repositories built with sub-optimal tree layouts or when dealing
with unreachable repositories. Used improperly, this option can generate
excessive load on repositories, cause synchronization to be interrupted by
firewalls, and generally creates create a public nuisance. Use with caution.

As of this writing, values in the range 2-4 are reasonably safe. Values above
10 have been known to cause problems.

rcynic can't really detect all of the possible problems created by excessive
values of this parameter, but if rcynic's report shows that both successful
retrivial and skipped retrieval from the same repository host, that's a pretty
good hint that something is wrong, and an excessive value here is a good first
guess as to the cause.

Default: 1

**** rsync-program ****

Path to the rsync program.

Default: rsync, but you should probably set this variable rather than just
trusting the PATH environment variable to be set correctly.

**** log-level ****

Same as -l option on command line. Command line setting overrides config file
setting.

Default: log_log_err

**** use-syslog ****

Same as -s option on command line. Command line setting overrides config file
setting.

Values: true or false.

Default: false

**** use-stderr ****

Same as -e option on command line. Command line setting overrides config file
setting.

Values: true or false.

Default: false, but if neither use-syslog nor use-stderr is set, log output
goes to stderr.

**** syslog-facility ****

Syslog facility to use.

Default: local0

**** syslog-priority-xyz ****

(where xyz is an rcynic logging level, above)

Override the syslog priority value to use when logging messages at this rcynic
level.

Defaults:

syslog-priority-log_sys_err   err

syslog-priority-log_usage_err err

syslog-priority-log_data_err  notice

syslog-priority-log_telemetry info

syslog-priority-log_verbose   info

syslog-priority-log_debug     debug

**** jitter ****

Startup jitter interval, same as -j option on command line. Jitter interval,
specified in number of seconds. rcynic will pick a random number within the
interval from zero to this value, and will delay for that many seconds on
startup. The purpose of this is to spread the load from large numbers of rcynic
clients all running under cron with synchronized clocks, in particular to avoid
hammering the global RPKI rsync servers into the ground at midnight UTC.

Default: 600

**** lockfile ****

Name of lockfile, or empty for no lock. If you run rcynic directly under cron,
you should use this parameter to set a lockfile so that successive instances of
rcynic don't stomp on each other. If you run rcynic under rcynic-cron, you
don't need to touch this, as rcynic-cron maintains its own lock.

Default: no lock

**** xml-summary ****

Enable output of a per-host summary at the end of an rcynic run in XML format.

Value: filename to which XML summary should be written; "-" will send XML
summary to standard output.

Default: no XML summary.

**** allow-stale-crl ****

Allow use of CRLs which are past their nextUpdate timestamp. This is usually
harmless, but since there are attack scenarios in which this is the first
warning of trouble, it's configurable.

Values: true or false.

Default: true

**** prune ****

Clean up old files corresponding to URIs that rcynic did not see at all during
this run. rcynic invokes rsync with the --delete option to clean up old objects
from collections that rcynic revisits, but if a URI changes so that rcynic
never visits the old collection again, old files will remain in the local
mirror indefinitely unless you enable this option.

Note: Pruning only happens when run-rsync is true. When the run-rsync option is
false, pruning is not done regardless of the setting of the prune option
option.

Values: true or false.

Default: true

**** allow-stale-manifest ****

Allow use of manifests which are past their nextUpdate timestamp. This is
probably harmless, but since it may be an early warning of problems, it's
configurable.

Values: true or false.

Default: true

**** require-crl-in-manifest ****

Reject publication point if manifest doesn't list the CRL that covers the
manifest EE certificate.

Values: true or false.

Default: false

**** allow-object-not-in-manifest ****

Allow use of otherwise valid objects which are not listed in the manifest. This
is not supposed to happen, but is probably harmless.

Enabling this does, however, often result in noisier logs, as it increases the
chance that rcynic will attempt to validate data which a CA removed from the
manifest but did not completely remove and revoke from the repository.

Values: true or false

Default: false

**** allow-digest-mismatch ****

Allow use of otherwise valid objects which are listed in the manifest with a
different digest value.

You probably don't want to touch this.

Values: true or false

Default: true

**** allow-crl-digest-mismatch ****

Allow processing to continue on a publication point whose manifest lists a
different digest value for the CRL than the digest of the CRL we have in hand.

You probably don't want to touch this.

Values: true or false

Default: true

**** allow-non-self-signed-trust-anchor ****

Experimental. Attempts to work around OpenSSL's strong preference for self-
signed trust anchors.

We're not going to explain this one in any further detail. If you really want
to know what it does, Use The Source, Luke.

Do not even consider enabling this option unless you are intimately familiar
with both X.509 and the internals of OpenSSL's X509_verify_cert() function and
really know what you are doing.

Values: true or false.

Default: false

**** run-rsync ****

Whether to run rsync to fetch data. You don't generally want to change this
except when building complex topologies where rcynic running on one set of
machines acts as aggregators for another set of validators. A large ISP might
want to build such a topology so that they could have a local validation cache
in each POP while minimizing load on the global repository system and
maintaining some degree of internal consistancy between POPs. In such cases,
one might want the rcynic instances in the POPs to validate data fetched from
the aggregators via an external process, without the POP rcynic instances
attempting to fetch anything themselves.

Values: true or false.

Default: true

**** use-links ****

Whether to use hard links rather than copying valid objects from the
unauthenticated to authenticated tree. Using links is slightly more fragile
(anything that stomps on the unauthenticated file also stomps on the
authenticated file) but is a bit faster and reduces the number of inodes
consumed by a large data collection. At the moment, copying is the default
behavior, but this may change in the future.

Values: true or false.

Default: false

**** rsync-early ****

Whether to force rsync to run even when we have a valid manifest for a
particular publication point and its nextUpdate time has not yet passed.

This is an experimental feature, and currently defaults to true, which is the
old behavior (running rsync regardless of whether we have a valid cached
manifest). This default may change once we have more experience with rcynic's
behavior when run with this option set to false.

Skipping the rsync fetch when we already have a valid cached manifest can
significantly reduce the total number of rsync connections we need to make, and
significantly reduce the load that each validator places on the authoritative
publication servers. As with any caching scheme, however, there are some
potential problems involved with not fetching the latest data, and we don't yet
have enough experience with this option to know how this will play out in
practice, which is why this is still considered experimental.

Values: true or false

Default: true (but may change in the future)

**** trust-anchor ****

Specify one RPKI trust anchor, represented as a local file containing an X.509
certificate in DER format. Value of this option is the pathname of the file.

No default.

**** trust-anchor-locator ****

Specify one RPKI trust anchor locator, represented as a local file in the
format specified in RFC-6490. This a simple text format containing an rsync URI
and the RSA public key of the X.509 object specified by the URI; the first line
of the file is the URI, the remainder is the public key in Base64 encoded DER
format.

Value of this option is the pathname of the file.

No default.

**** trust-anchor-directory ****

Specify a directory containing trust anchors, trust anchor locators, or both.
Trust anchors in such a directory must have filenames ending in ".cer"; trust
anchor locators in such a directory must have names ending in ".tal"; any other
files will be skipped.

This directive is an alternative to using the trust-anchor and trust-anchor-
locator` directives. This is probably easier to use than the other trust anchor
directives when dealing with a collection of trust anchors. This may change on
that promised day when we have only a single global trust anchor to deal with,
but we're not there yet.

No default.

***** Post-processing rcynic's XML output *****

The distribution includes several post-processors for the XML output rcynic
writes describing the actions it has taken and the validation status of the
objects it has found.

**** rcynic-html ****

rcynic-html converts rcynic's XML output into a collection of HTML pages
summarizing the results, noting problems encountered, and showing some history
of rsync transfer times and repository object counts in graphical form.

rcynic-cron runs rcynic-html automatically, immediately after running rcynic.
If for some reason you need to run rcynic-html by hand, the command syntax is:

  $ rcynic-html rcynic.xml /web/server/directory/

rcynic-html will write a collection of HTML and image files to the specified
output directory, along with a set of RRD databases. rcynic-html will create
the output directory if necessary.

rcynic-html requires `rrdtool`, a specialized database and graphing engine
designed for this sort of work. You can run rcynic-html without rrdtool by
giving it the --no-show-graphs option, but the result won't be as useful.

rcynic-html gets its idea of where to find the rrdtool program from autoconf,
which usually works. If for some reason it doesn't work in your environment,
you will need to tell rcynic-html where to find rrdtool, using the --rrdtool-
binary option:

  $ rcynic-html --rrdtoolbinary /some/where/rrdtool rcynic.xml /web/server/
  directory/

**** rcynic.xsl ****

rcynic.xsl was an earlier attempt at the same kind of HTML output as rcynic-
html generates. XSLT was a convenient language for our initial attempts at
this, but as the processing involved got more complex, it became obvious that
we needed a general purpose programming language.

If for some reason XSLT works better in your environment than Python, you might
find this stylesheet to be a useful starting point, but be warned that it's
significantly slower than rcynic-html, lacks many features, and is no longer
under development.

**** rcynic-text ****

rcynic-text provides a quick flat text summary of validation results. This is
useful primarily in test scripts (smoketest uses it).

Usage:

  $ rcynic-text rcynic.xml

**** validation_status ****

validation_status provides a flat text translation of the detailed validation
results. This is useful primarily for checking the detailed status of some
particular object or set of objects, perhaps using a program like grep or awk
to filter validation_status's output.

Usage:

  $ validation_status rcynic.xml
  $ validation_status rcynic.xml | fgrep rpki.misbehaving.org
  $ validation_status rcynic.xml | fgrep object_rejected

**** rcynic-svn ****

rcynic-svn is a tool for archiving rcynic's results in a Subversion repository.
rcynic-svn is not integrated into rcynic-cron, because this is not something
that every relying party is going to want to do. However, for relying parties
who want to analyze rcynic's output over a long period of time, rcynic-svn may
provide a useful starting point starting point.

To use rcynic-svn, you first must set up a Subversion repository and check out
a working directory:

  $ svnadmin create /some/where/safe/rpki-archive
  $ svn co file:///some/where/safe/rpki-archive /some/where/else/rpki-archive

The name can be anything you like, in this example we call it "rpki-archive".
The above sequence creates the repository, then checks out an empty working
directory /some/where/else/rpki-archive.

The repository does not need to be on the same machine as the working
directory, but it probably should be for simplicity unless you have some strong
need to put it elsewhere.

Once you have the repository and working directory set up, you need to arrange
for rcynic-svn to be run after each rcynic run whose results you want to
archive. One way to do this would be to run rcynic-svn in the same cron job as
rcynic-cron, immediately after rcynic-cron and specifying the same lock file
that rcynic-cron uses.

Sample usage, assuming that rcynic's data is in the usual place:

  $ rcynic-svn --lockfile /var/rcynic/data/lock   \
          /var/rcynic/data/authenticated          \
          /var/rcynic/data/unauthenticated        \
          /var/rcynic/data/rcynic.xml             \
          /some/where/else/rpki-archive

where the last argument is the name of the Subversion working directory and the
other arguments are the names of those portions of rcynic's output which you
wish to archive. Generally, the above set (authenticated, unauthenticated, and
rcynic.xml) are the ones you want, but feel free to experiment.