Drop in documentation extracted from wiki.rpki.net. See README for details.

1 files changed, 0 insertions, 634 deletions

diff --git a/doc/doc.RPKI.RP.rcynic b/doc/doc.RPKI.RP.rcynic
deleted file mode 100644
index 4bd95ae1..00000000
--- a/doc/doc.RPKI.RP.rcynic
+++ /dev/null
@@ -1,634 +0,0 @@
-****** rcynic RPKI validator ******
-
-rcynic is the core RPKI relying party tool, and is the code which performs the
-actual RPKI validation. Most of the other relying party tools just use rcynic's
-output.
-
-The name is short for "cynical rsync", because rcynic's task involves an
-interleaved process of rsync retrieval and RPKI validation.
-
-This code was developed on FreeBSD, and has been tested most heavily on FreeBSD
-versions 6-STABLE through 8-STABLE. It is also known to work on Ubuntu (12.04
-LTS), Debian (Wheezy) and Mac OS X (Snow Leopard). In theory it should run on
-any reasonably POSIX-like system. As far as we know, rcynic does not use any
-seriously non-portable features, but neither have we done a POSIX reference
-manual lookup for every function call. Please report any portability problems.
-
-***** Don't panic *****
-
-rcynic has a lot of options, but it attempts to choose reasonable defaults
-where possible. The installation process will create a basic working rcynic
-configuration for you and arrange for this to run hourly under cron. If all
-goes well, this should "just work".
-
-rcynic has the ability to do all of its work in a chroot jail. This used to be
-the default configuration, but integrating this properly with platform-specific
-packaging systems (FreeBSD ports, apt-get on Ubuntu and Debian, etc) proved
-impractical. You can still get this behavior if you need it, by installing from
-source and using the --enable-rcynic-jail option to ./configure.
-
-The default configuration set up by make install and the various packaging
-systems will run rcynic under cron using the rcynic-cron wrapper script. See
-the instructions for setting up your own cron jobs if you need something more
-complicated; also see the instructions for setting up hierarchical rsync if you
-need to build a complex topology of rcynic validators.
-
-***** Overview *****
-
-rcynic depends heavily on the OpenSSL libcrypto library, and requires a
-reasonably current version of OpenSSL with both RFC 3779 and CMS support.
-
-rcynic expects all certificates, CRLs, and CMS objects to be in DER format.
-rcynic stores its database using filenames derived from the RPKI rsync URIs at
-which the data are published.
-
-All configuration is via an OpenSSL-style configuration file, except for
-selection of the name of the configuration file itself. A few other parameters
-can also be set from the command line. The default name for the configuration
-is "rcynic.conf"; you can override this with the -c option on the command line.
-The configuration file uses OpenSSL's configuration file syntax, and you can
-set OpenSSL library configuration paramaters (eg, "engine" settings) in the
-config file as well. rcynic's own configuration parameters are in a section
-called "[rcynic]".
-
-Most configuration parameters are optional and have defaults which should do
-something reasonable if you are running rcynic in a test directory. If you're
-running rcynic as a system program, perhaps under cron via the rcynic-cron
-script, you'll want to set additional parameters to tell rcynic where to find
-its data and where to write its output (the installation process sets these
-parameters for you). The configuration file itself, however, is not optional.
-In order for rcynic to do anything useful, your configuration file MUST at
-minimum tell rcynic where to find one or more RPKI trust anchors or trust
-anchor locators (TALs).
-
-**** Trust anchors ****
-
-* To specify a trust anchor, use the trust-anchor directive to name the local
-  file containing the trust anchor.
-
-* To specify a trust anchor locator (TAL), use the trust-anchor-locator
-  directive to name a local file containing the trust anchor locator.
-
-* To specify a directory containing trust anchors or trust anchor locators, use
-  the trust-anchor-directory directive to name the directory. Files in the
-  specified directory with names ending in ".cer" will be processed as trust
-  anchors, while files with names ending in ".tal" will be processed as trust
-  anchor locators.
-
-You may use a combination of these methods if necessary.
-
-Trust anchors are represented as DER-formatted X.509 self-signed certificate
-objects, but in practice trust anchor locators are more common, as they reduce
-the amount of locally configured data to the bare minimum and allow the trust
-anchor itself to be updated without requiring reconfiguration of validators
-like rcynic. A trust anchor locator is a file in the format specified in RFC-
-6490, consisting of the rsync URI of the trust anchor followed by the Base64
-encoding of the trust anchor's public key.
-
-Strictly speaking, trust anchors do not need to be self-signed, but many
-programs (including OpenSSL) assume that trust anchors will be self-signed. See
-the allow-non-self-signed-trust-anchor configuration option if you need to use
-a non-self-signed trust anchor, but be warned that the results, while
-technically correct, may not be useful.
-
-See the make-tal.sh script in this directory if you need to generate your own
-TAL file for a trust anchor.
-
-As of this writing, there still is no single global trust anchor for the RPKI
-system, so you have to provide separate trust anchors for each Regional
-Internet Registry (RIR) which is publishing RPKI data. The installation process
-installs the ones it knows about.
-
-Example of a minimal config file specifying nothing but trust anchor locators:
-
-  [rcynic]
-
-  trust-anchor-locator.0 = trust-anchors/apnic.tal
-  trust-anchor-locator.1 = trust-anchors/ripe.tal
-  trust-anchor-locator.2 = trust-anchors/afrinic.tal
-  trust-anchor-locator.3 = trust-anchors/lacnic.tal
-
-Eventually, this should all be collapsed into a single trust anchor, so that
-relying parties don't need to sort this out on their own, at which point the
-above configuration could become something like:
-
-  [rcynic]
-
-  trust-anchor-locator = trust-anchors/iana.tal
-
-**** Output directories ****
-
-By default, rcynic uses two writable directory trees:
-
-unauthenticated::
-
-     Raw data fetched via rsync. In order to take full advantage of
-     rsync's optimized transfers, you should preserve and reuse this
-     directory across rcynic runs, so that rcynic need not re-fetch data
-     that have not changed.
-
-authenticated::
-
-     Data which rcynic has checked. This is the real output of the
-     validation process.
-
-authenticated is really a symbolic link to a directory with a name of the form
-"authenticated.<timestamp>", where <timestamp> is an ISO 8601 timestamp like
-2001-04-01T01:23:45Z. rcynic creates a new timestamped directory every time it
-runs, and moves the symbolic link as an atomic operation when the validation
-process completes. The intent is that authenticated always points to the most
-recent usable validation results, so that programs which use rcynic's output
-don't need to worry about whether an rcynic run is in progress.
-
-rcynic installs trust anchors specified via the trust-anchor-locator directive
-in the unauthenticated tree just like any other fetched object, and copies them
-into the authenticated trees just like any other object once they pass rcynic's
-checks.
-
-rcynic copies trust anchors specified via the trust-anchor directive into the
-top level directory of the authenticated tree with filenames of the form
-<xxxxxxxx>.<n>.cer, where <xxxxxxxx> and <n> are the OpenSSL object name hash
-and index within the resulting virtual hash bucket, respectively. These are the
-same values that OpenSSL's c_hash Perl script would produce. The reason for
-this naming scheme is that these trust anchors, by definition, are not fetched
-automatically, and thus do not really have publication URIs in the sense that
-every other object in these trees do. So rcynic uses a naming scheme which
-insures:
-
-* that each trust anchor has a unique name within the output tree and
-
-* that trust anchors cannot be confused with certificates: trust anchors always
-  go in the top level of the tree, data fetched via rsync always go in
-  subdirectories.
-
-Trust anchors and trust anchor locators taken from the directory named by the
-trust-anchor-directory directive will follow the same naming scheme trust
-anchors and trust anchor locators specified via the trust-anchor and trust-
-anchor-locator directives, respectively.
-
-***** Usage and configuration *****
-
-**** Logging levels ****
-
-rcynic has its own system of logging levels, similar to what syslog() uses, but
-customized to the specific task rcynic performs.
-
-log_sys_err   Error from operating system or library
-
-log_usage_err Bad usage (local configuration error)
-
-log_data_err  Bad data (broken certificates or CRLs)
-
-log_telemetry Normal chatter about rcynic's progress
-
-log_verbose   Extra verbose chatter
-
-log_debug     Only useful when debugging
-
-**** Command line options ****
-
--c configfile Path to configuration file (default: rcynic.conf)
-
--l loglevel   Logging level (default: log_data_err)
-
--s            Log via syslog
-
--e            Log via stderr when also using syslog
-
--j            Start-up jitter interval (see below; default: 600)
-
--V            Print rcynic's version to standard output and exit
-
--x            Path to XML "summary" file (see below; no default)
-
-***** Configuration file reference *****
-
-rcynic uses the OpenSSL libcrypto configuration file mechanism. All libcrypto
-configuration options (eg, for engine support) are available. All rcynic-
-specific options are in the "[rcynic]" section. You MUST have a configuration
-file in order for rcynic to do anything useful, as the configuration file is
-the only way to list your trust anchors.
-
-**** authenticated ****
-
-Path to output directory (where rcynic should place objects it has been able to
-validate).
-
-Default: rcynic-data/authenticated
-
-**** unauthenticated ****
-
-Path to directory where rcynic should store unauthenticatd data retrieved via
-rsync. Unless something goes horribly wrong, you want rcynic to preserve and
-reuse this directory across runs to minimize the network traffic necessary to
-bring your repository mirror up to date.
-
-Default: rcynic-data/unauthenticated
-
-**** rsync-timeout ****
-
-How long (in seconds) to let rsync run before terminating the rsync process, or
-zero for no timeout. You want this timeout to be fairly long, to avoid
-terminating rsync connections prematurely. It's present to let you defend
-against evil rsync server operators who try to tarpit your connection as a form
-of denial of service attack on rcynic.
-
-Default: 300
-
-**** max-parallel-fetches ****
-
-Upper limit on the number of copies of rsync that rcynic is allowed to run at
-once. Used properly, this can speed up synchronization considerably when
-fetching from repositories built with sub-optimal tree layouts or when dealing
-with unreachable repositories. Used improperly, this option can generate
-excessive load on repositories, cause synchronization to be interrupted by
-firewalls, and generally creates create a public nuisance. Use with caution.
-
-As of this writing, values in the range 2-4 are reasonably safe. Values above
-10 have been known to cause problems.
-
-rcynic can't really detect all of the possible problems created by excessive
-values of this parameter, but if rcynic's report shows that both successful
-retrivial and skipped retrieval from the same repository host, that's a pretty
-good hint that something is wrong, and an excessive value here is a good first
-guess as to the cause.
-
-Default: 1
-
-**** rsync-program ****
-
-Path to the rsync program.
-
-Default: rsync, but you should probably set this variable rather than just
-trusting the PATH environment variable to be set correctly.
-
-**** log-level ****
-
-Same as -l option on command line. Command line setting overrides config file
-setting.
-
-Default: log_log_err
-
-**** use-syslog ****
-
-Same as -s option on command line. Command line setting overrides config file
-setting.
-
-Values: true or false.
-
-Default: false
-
-**** use-stderr ****
-
-Same as -e option on command line. Command line setting overrides config file
-setting.
-
-Values: true or false.
-
-Default: false, but if neither use-syslog nor use-stderr is set, log output
-goes to stderr.
-
-**** syslog-facility ****
-
-Syslog facility to use.
-
-Default: local0
-
-**** syslog-priority-xyz ****
-
-(where xyz is an rcynic logging level, above)
-
-Override the syslog priority value to use when logging messages at this rcynic
-level.
-
-Defaults:
-
-syslog-priority-log_sys_err   err
-
-syslog-priority-log_usage_err err
-
-syslog-priority-log_data_err  notice
-
-syslog-priority-log_telemetry info
-
-syslog-priority-log_verbose   info
-
-syslog-priority-log_debug     debug
-
-**** jitter ****
-
-Startup jitter interval, same as -j option on command line. Jitter interval,
-specified in number of seconds. rcynic will pick a random number within the
-interval from zero to this value, and will delay for that many seconds on
-startup. The purpose of this is to spread the load from large numbers of rcynic
-clients all running under cron with synchronized clocks, in particular to avoid
-hammering the global RPKI rsync servers into the ground at midnight UTC.
-
-Default: 600
-
-**** lockfile ****
-
-Name of lockfile, or empty for no lock. If you run rcynic directly under cron,
-you should use this parameter to set a lockfile so that successive instances of
-rcynic don't stomp on each other. If you run rcynic under rcynic-cron, you
-don't need to touch this, as rcynic-cron maintains its own lock.
-
-Default: no lock
-
-**** xml-summary ****
-
-Enable output of a per-host summary at the end of an rcynic run in XML format.
-
-Value: filename to which XML summary should be written; "-" will send XML
-summary to standard output.
-
-Default: no XML summary.
-
-**** allow-stale-crl ****
-
-Allow use of CRLs which are past their nextUpdate timestamp. This is usually
-harmless, but since there are attack scenarios in which this is the first
-warning of trouble, it's configurable.
-
-Values: true or false.
-
-Default: true
-
-**** prune ****
-
-Clean up old files corresponding to URIs that rcynic did not see at all during
-this run. rcynic invokes rsync with the --delete option to clean up old objects
-from collections that rcynic revisits, but if a URI changes so that rcynic
-never visits the old collection again, old files will remain in the local
-mirror indefinitely unless you enable this option.
-
-Note: Pruning only happens when run-rsync is true. When the run-rsync option is
-false, pruning is not done regardless of the setting of the prune option
-option.
-
-Values: true or false.
-
-Default: true
-
-**** allow-stale-manifest ****
-
-Allow use of manifests which are past their nextUpdate timestamp. This is
-probably harmless, but since it may be an early warning of problems, it's
-configurable.
-
-Values: true or false.
-
-Default: true
-
-**** require-crl-in-manifest ****
-
-Reject publication point if manifest doesn't list the CRL that covers the
-manifest EE certificate.
-
-Values: true or false.
-
-Default: false
-
-**** allow-object-not-in-manifest ****
-
-Allow use of otherwise valid objects which are not listed in the manifest. This
-is not supposed to happen, but is probably harmless.
-
-Enabling this does, however, often result in noisier logs, as it increases the
-chance that rcynic will attempt to validate data which a CA removed from the
-manifest but did not completely remove and revoke from the repository.
-
-Values: true or false
-
-Default: false
-
-**** allow-digest-mismatch ****
-
-Allow use of otherwise valid objects which are listed in the manifest with a
-different digest value.
-
-You probably don't want to touch this.
-
-Values: true or false
-
-Default: true
-
-**** allow-crl-digest-mismatch ****
-
-Allow processing to continue on a publication point whose manifest lists a
-different digest value for the CRL than the digest of the CRL we have in hand.
-
-You probably don't want to touch this.
-
-Values: true or false
-
-Default: true
-
-**** allow-non-self-signed-trust-anchor ****
-
-Experimental. Attempts to work around OpenSSL's strong preference for self-
-signed trust anchors.
-
-We're not going to explain this one in any further detail. If you really want
-to know what it does, Use The Source, Luke.
-
-Do not even consider enabling this option unless you are intimately familiar
-with both X.509 and the internals of OpenSSL's X509_verify_cert() function and
-really know what you are doing.
-
-Values: true or false.
-
-Default: false
-
-**** run-rsync ****
-
-Whether to run rsync to fetch data. You don't generally want to change this
-except when building complex topologies where rcynic running on one set of
-machines acts as aggregators for another set of validators. A large ISP might
-want to build such a topology so that they could have a local validation cache
-in each POP while minimizing load on the global repository system and
-maintaining some degree of internal consistancy between POPs. In such cases,
-one might want the rcynic instances in the POPs to validate data fetched from
-the aggregators via an external process, without the POP rcynic instances
-attempting to fetch anything themselves.
-
-Values: true or false.
-
-Default: true
-
-**** use-links ****
-
-Whether to use hard links rather than copying valid objects from the
-unauthenticated to authenticated tree. Using links is slightly more fragile
-(anything that stomps on the unauthenticated file also stomps on the
-authenticated file) but is a bit faster and reduces the number of inodes
-consumed by a large data collection. At the moment, copying is the default
-behavior, but this may change in the future.
-
-Values: true or false.
-
-Default: false
-
-**** rsync-early ****
-
-Whether to force rsync to run even when we have a valid manifest for a
-particular publication point and its nextUpdate time has not yet passed.
-
-This is an experimental feature, and currently defaults to true, which is the
-old behavior (running rsync regardless of whether we have a valid cached
-manifest). This default may change once we have more experience with rcynic's
-behavior when run with this option set to false.
-
-Skipping the rsync fetch when we already have a valid cached manifest can
-significantly reduce the total number of rsync connections we need to make, and
-significantly reduce the load that each validator places on the authoritative
-publication servers. As with any caching scheme, however, there are some
-potential problems involved with not fetching the latest data, and we don't yet
-have enough experience with this option to know how this will play out in
-practice, which is why this is still considered experimental.
-
-Values: true or false
-
-Default: true (but may change in the future)
-
-**** trust-anchor ****
-
-Specify one RPKI trust anchor, represented as a local file containing an X.509
-certificate in DER format. Value of this option is the pathname of the file.
-
-No default.
-
-**** trust-anchor-locator ****
-
-Specify one RPKI trust anchor locator, represented as a local file in the
-format specified in RFC-6490. This a simple text format containing an rsync URI
-and the RSA public key of the X.509 object specified by the URI; the first line
-of the file is the URI, the remainder is the public key in Base64 encoded DER
-format.
-
-Value of this option is the pathname of the file.
-
-No default.
-
-**** trust-anchor-directory ****
-
-Specify a directory containing trust anchors, trust anchor locators, or both.
-Trust anchors in such a directory must have filenames ending in ".cer"; trust
-anchor locators in such a directory must have names ending in ".tal"; any other
-files will be skipped.
-
-This directive is an alternative to using the trust-anchor and trust-anchor-
-locator` directives. This is probably easier to use than the other trust anchor
-directives when dealing with a collection of trust anchors. This may change on
-that promised day when we have only a single global trust anchor to deal with,
-but we're not there yet.
-
-No default.
-
-***** Post-processing rcynic's XML output *****
-
-The distribution includes several post-processors for the XML output rcynic
-writes describing the actions it has taken and the validation status of the
-objects it has found.
-
-**** rcynic-html ****
-
-rcynic-html converts rcynic's XML output into a collection of HTML pages
-summarizing the results, noting problems encountered, and showing some history
-of rsync transfer times and repository object counts in graphical form.
-
-rcynic-cron runs rcynic-html automatically, immediately after running rcynic.
-If for some reason you need to run rcynic-html by hand, the command syntax is:
-
-  $ rcynic-html rcynic.xml /web/server/directory/
-
-rcynic-html will write a collection of HTML and image files to the specified
-output directory, along with a set of RRD databases. rcynic-html will create
-the output directory if necessary.
-
-rcynic-html requires `rrdtool`, a specialized database and graphing engine
-designed for this sort of work. You can run rcynic-html without rrdtool by
-giving it the --no-show-graphs option, but the result won't be as useful.
-
-rcynic-html gets its idea of where to find the rrdtool program from autoconf,
-which usually works. If for some reason it doesn't work in your environment,
-you will need to tell rcynic-html where to find rrdtool, using the --rrdtool-
-binary option:
-
-  $ rcynic-html --rrdtoolbinary /some/where/rrdtool rcynic.xml /web/server/
-  directory/
-
-**** rcynic.xsl ****
-
-rcynic.xsl was an earlier attempt at the same kind of HTML output as rcynic-
-html generates. XSLT was a convenient language for our initial attempts at
-this, but as the processing involved got more complex, it became obvious that
-we needed a general purpose programming language.
-
-If for some reason XSLT works better in your environment than Python, you might
-find this stylesheet to be a useful starting point, but be warned that it's
-significantly slower than rcynic-html, lacks many features, and is no longer
-under development.
-
-**** rcynic-text ****
-
-rcynic-text provides a quick flat text summary of validation results. This is
-useful primarily in test scripts (smoketest uses it).
-
-Usage:
-
-  $ rcynic-text rcynic.xml
-
-**** validation_status ****
-
-validation_status provides a flat text translation of the detailed validation
-results. This is useful primarily for checking the detailed status of some
-particular object or set of objects, perhaps using a program like grep or awk
-to filter validation_status's output.
-
-Usage:
-
-  $ validation_status rcynic.xml
-  $ validation_status rcynic.xml | fgrep rpki.misbehaving.org
-  $ validation_status rcynic.xml | fgrep object_rejected
-
-**** rcynic-svn ****
-
-rcynic-svn is a tool for archiving rcynic's results in a Subversion repository.
-rcynic-svn is not integrated into rcynic-cron, because this is not something
-that every relying party is going to want to do. However, for relying parties
-who want to analyze rcynic's output over a long period of time, rcynic-svn may
-provide a useful starting point starting point.
-
-To use rcynic-svn, you first must set up a Subversion repository and check out
-a working directory:
-
-  $ svnadmin create /some/where/safe/rpki-archive
-  $ svn co file:///some/where/safe/rpki-archive /some/where/else/rpki-archive
-
-The name can be anything you like, in this example we call it "rpki-archive".
-The above sequence creates the repository, then checks out an empty working
-directory /some/where/else/rpki-archive.
-
-The repository does not need to be on the same machine as the working
-directory, but it probably should be for simplicity unless you have some strong
-need to put it elsewhere.
-
-Once you have the repository and working directory set up, you need to arrange
-for rcynic-svn to be run after each rcynic run whose results you want to
-archive. One way to do this would be to run rcynic-svn in the same cron job as
-rcynic-cron, immediately after rcynic-cron and specifying the same lock file
-that rcynic-cron uses.
-
-Sample usage, assuming that rcynic's data is in the usual place:
-
-  $ rcynic-svn --lockfile /var/rcynic/data/lock   \
-          /var/rcynic/data/authenticated          \
-          /var/rcynic/data/unauthenticated        \
-          /var/rcynic/data/rcynic.xml             \
-          /some/where/else/rpki-archive
-
-where the last argument is the name of the Subversion working directory and the
-other arguments are the names of those portions of rcynic's output which you
-wish to archive. Generally, the above set (authenticated, unauthenticated, and
-rcynic.xml) are the ones you want, but feel free to experiment.