1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
|
$Id$
Rough plan for initial prototype RE.
1) Hack quick cert generator [done]
2) Hack xml-parse-test into something that can:
- Generate XML (just compose directly, probably)
- Encode XML in CMS using specified signing key and certs
- Write that to a destination file
- Pick up signed CMS from from a file
- Check CMS signatures and extract XML
- Validate (xmllint --relaxng) the XML
- Decode (xml.sax) the XML and dump the result
Log everything as we go.
data structures.
Schema is RNC (RelaxNG Compact) syntax, xmllint wants RNG (RelaxNG
XML) syntax, so use /usr/ports/textproc/trang to convert:
$ trang foo.rnc foo.rng
3) Whack (2) into something that does this over minimal HTTP.
4) [to be continued...]
Fun with SQL. Tim has given me a nice SQL schema for the objects we
worked out in Prague (superset of left-right protocol). Still
debugging that design, but broad outlines are clear.
Some question how to deal with in-memory data and SQL without going
mad. Am assuming RE is event driven. Current theory: pull in an
entire context (one row of the "self" table and everything that links
to that self_id), run this event with it, save out any changes before
this event terminates. Perhaps keep an LRU cache of the last few such
contexts in which we've run, to save loading every time (eg, receipt
of an answer to an up-down or left-right protocol query is a discrete
event, would be annoying to have to read everything back from SQL
again).
If we can work out a tree that spans one such memory context (almost
there), we can just walk that tree to handle pretty much all SQL.
Basic data structures we need for every object we're going to store in
SQL:
- Object type to hold the object (duh).
- List of children in the data structure tree.
- SQL SELECT expression (including variable bindings!) to pull all
rows matching blah blah out of SQL. Tricky bit here is variable
bindings, as otherwise the obvious handling would be a simple
method. Various possible ways to hack this in Python, perhaps just
lambda(x,y,z): """SQL blah blah""" % dict
Or perhaps this meta-tree is itself built from a Python class
(probably should be) and these lambdas turn into methods of the
meta-tree. Or something.
- SQL to INSERT ...
- SQL to UPDATE ...
- SQL to DROP ...
At this point it's starting to look like the pdu objects in
rpki.left_right are a subclass of a more general class of objects that
live in SQL (duh, again). This makes sense as left-right protocol was
designed to let us frob a subset of the rpki database on the wire.
Need to check whether current rpki.left_right objects map cleanly to
data structures we're saving in SQL. They should, but check anyway.
Hmm, it's more complicated than that. Most of left_right maps well,
but portions (eg, <list_resources/>) are just commands and do not.
Mixin? We have a varient of this same problem with the mixins I'm
already using in irbe-cli.py.
Also need to think about up-down protocol, currently separate but some
of these objects might map there too. This is why I originally
intended to code all of these as separate object hierarchies even
though in some cases they look awfully similar. Hmm, implement as
separate hierarchies with sideways conversion methods? Still feels
wrong, but so does turning this into one big interconnected mess.
Ok, up-down XML is all structured as commands, data is all attributes
and (occasionally) element content, so not very close match to db.
So this appears to break out fairly cleanly: objects shown in blue in
the repository-engine-objects.pdf picture are both left-right PDU
objects and database objects, objects shown in green are only database
objects, and left-right PDU objects not shown in that picture are just
left-right PDUs. Or something close to this, details need checking.
So this does look like some kind of mixin would be appropriate.
|
