feat: rewrite docs
This commit is contained in:
140 files changed
+4555
-701
No files matched your search
+42
-9
@@ -1,24 +1,57 @@
|
||||
# Tools
|
||||
|
||||
Small programs that build and check the repository, and one for loading packages into SQLite.
|
||||
They use only the Python standard library and are licensed Apache-2.0.
|
||||
This page is for anyone who wants to run the repository's checks, rebuild its examples or load a document into a database.
|
||||
Newcomers get each term defined where it first appears.
|
||||
Implementers get the commands and the traps.
|
||||
|
||||
These programs are the repository's reference implementation.
|
||||
They apply every rule of the specification, so the rest of the repository can be checked against them.
|
||||
Two build and check the repository, and one loads packages into SQLite, a database stored in a single file.
|
||||
They use only the Python standard library (Python 3.10 or later [CHECK: stated in README.md; pixi.toml pins 3.14 and I could only run on 3.13]) and are licensed Apache-2.0.
|
||||
|
||||
A package is the folder of files that holds one document, such as `fixtures/packages/hale2026harmonic`.
|
||||
A vector is a test case: an input and the exact output every implementation must compute (see [the vectors](../vectors/README.md)).
|
||||
A fixture is an example file or package under `fixtures/`.
|
||||
|
||||
| Program | Does |
|
||||
| --- | --- |
|
||||
| `check.py` | Checks everything: schemas, records, packages, the broken packages, every vector, a byte-for-byte rebuild, the SQL round trip and the prose. `--quick` skips the rebuild. |
|
||||
| `check.py` | Checks everything: schemas, records, packages, the broken packages, every vector, a byte-for-byte rebuild, the SQL (a database query language) round trip and the prose. `--quick` skips the rebuild. |
|
||||
| `build.py` | Builds `fixtures/` and `vectors/` from the hand-written sources in `fixtures/src/` and the cases it defines. `--out DIR` builds into another folder. |
|
||||
| `load.py` | Loads packages into a SQLite database with the tables of `sql/`: `python3 tools/load.py --sqlite lab.db PACKAGE…`. |
|
||||
| `oracle/` | A compact implementation of every rule, which `build.py` uses to compute expected values and `check.py` uses to recompute them. |
|
||||
| `load.py` | Loads packages into a SQLite database with the tables of `sql/`: `python3 tools/load.py --sqlite lab.db PACKAGE [PACKAGE ...]`. |
|
||||
| `oracle/` | A compact implementation of every rule. `build.py` uses it to compute expected values, and `check.py` uses it to recompute them. |
|
||||
|
||||
## When to run what
|
||||
|
||||
Each program is also a pixi task, so `pixi run check`, `pixi run quick`, `pixi run build` and `pixi run load` work from anywhere in the repository; `pixi run check` lints the Markdown first.
|
||||
Each program is also a pixi task.
|
||||
Pixi sets up a pinned Python and the other packages the repository uses.
|
||||
The commands `pixi run check`, `pixi run quick`, `pixi run build` and `pixi run load` then work from anywhere in the repository [CHECK: pixi is not installed here, so the pixi forms were not run]. `pixi run check` lints the Markdown first.
|
||||
|
||||
Run the programs directly from the repository root if you do not use pixi:
|
||||
|
||||
```sh
|
||||
python3 tools/check.py # everything
|
||||
python3 tools/check.py --quick # everything except the rebuild
|
||||
python3 tools/build.py --out /tmp/rebuilt
|
||||
python3 tools/load.py --sqlite lab.db fixtures/packages/hale2026harmonic
|
||||
```
|
||||
|
||||
A passing check ends with `All checks pass.`
|
||||
The load command prints `hale2026harmonic: 75 nodes`, one line per package, and writes `lab.db` with a folder `lab.db.assets/` beside it for files that have no table of their own.
|
||||
`build.py` has no help option and ignores flags it does not know, so it rebuilds `fixtures/` and `vectors/` in place unless you pass `--out`.
|
||||
|
||||
Run `pixi run check` before every commit.
|
||||
Run `pixi run build` after changing a fixture source, a rule in `oracle/`, or a case in `build.py`, then commit the regenerated files with the change; the check fails if they are out of date.
|
||||
Run `pixi run build` after you change a fixture source, a rule in `oracle/` or a case in `build.py`, and commit the regenerated files with the change.
|
||||
The check rebuilds into a temporary folder and compares every file byte for byte, so it fails on out-of-date files.
|
||||
|
||||
## About the oracle
|
||||
|
||||
An oracle is a reference computation: a slow, plain program whose answers count as the expected ones.
|
||||
It is not a production reader or writer.
|
||||
The oracle exists so that no expected value in this repository is typed by hand.
|
||||
It favors clarity over speed, its Ed25519 is the slow reference code of RFC 8032, and it is not a library to build on: implementations follow the [specification](../spec/README.md) and test themselves against the [vectors](../vectors/README.md).
|
||||
Its JSON Schema validator supports exactly the keyword subset the schemas use, and refuses any schema that uses more.
|
||||
|
||||
It favors clarity over speed.
|
||||
Its Ed25519 (a digital signature scheme) is the slow reference code of RFC 8032.
|
||||
Its validator for JSON Schema (a format that describes the shape of JSON data) supports exactly the keyword subset the schemas use and refuses any schema that uses more.
|
||||
|
||||
Do not build on it as a library.
|
||||
Implementations follow the [specification](../spec/README.md) and test themselves against the [vectors](../vectors/README.md).
|
||||
+4
-4
@@ -39,7 +39,7 @@ def H(s):
|
||||
|
||||
|
||||
def secret(name):
|
||||
return hashlib.sha256(("varve-spec fixture key: " + name).encode()).digest()
|
||||
return hashlib.sha256(("manuspec fixture key: " + name).encode()).digest()
|
||||
|
||||
|
||||
KEYS = {
|
||||
@@ -48,7 +48,7 @@ KEYS = {
|
||||
"ben-spot": {"name": "Ben Example", "purposes": ["spot-check"]},
|
||||
"cara": {"name": "Cara Example", "purposes": ["sign"]},
|
||||
}
|
||||
TEAM_SECRET = hashlib.sha256(b"varve-spec fixture team secret: smithlab").digest()
|
||||
TEAM_SECRET = hashlib.sha256(b"manuspec fixture team secret: smithlab").digest()
|
||||
|
||||
|
||||
def roster():
|
||||
@@ -513,8 +513,8 @@ def package_files(src, key, names, side, final, records, receipts=None, roster_r
|
||||
files["roster.json"] = (jcs.dumps(roster_rec) + "\n").encode()
|
||||
files["witnesses.json"] = (jcs.dumps({"witnesses": witnesses}) + "\n").encode()
|
||||
files["provenance.json"] = (jcs.dumps({
|
||||
"tools": [{"name": "varve-spec tools/build.py", "version": VERSION}],
|
||||
"stages": [{"stage": "hand", "processor": "local", "tool": "varve-spec tools/build.py"}],
|
||||
"tools": [{"name": "manuspec tools/build.py", "version": VERSION}],
|
||||
"stages": [{"stage": "hand", "processor": "local", "tool": "manuspec tools/build.py"}],
|
||||
"note": "Built by hand for the conformance suite; every reading is synthetic."}) + "\n").encode()
|
||||
levels = RC.effective_levels(nodes, vals, receipts or [], ROSTER_BY_ID if roster_rec else {})
|
||||
d = dict(src["document"])
|
||||
|
||||
Reference in new issue
Block a user