refactor: move the reference implementation to src/paperspec
The oracle had two import names, `tools.oracle` for the tools and `paperspec` for other repositories, which could load the same code twice in one process and needed a gate keyed on the module name. It now lives in the standard src layout under its one name. - git mv tools/oracle -> src/paperspec; the 22 rule modules are unchanged. - The tools import `paperspec`. `tools/__init__.py` now holds the Python 3.14 and cryptography/jsonschema check and puts src/ first on sys.path, so the tools run this checkout's code with or without pixi. The package's own __init__ checks nothing and exposes __version__. - pyproject.toml: version from git tags via setuptools-scm (guess-next-dev, node-and-timestamp, src/paperspec/_version.py, gitignored). The packages stay listed rather than found because schemas/, sql/ and vectors/ stay at the root: the spec links to them and the site serves the schemas at their $id URLs. - pixi.toml: a `dev` feature and environment with setuptools, setuptools-scm and python-build, and `pixi run -e dev wheel`. - check.py scans src/paperspec for rule IDs; README, CONTRIBUTING, the FAQ, the overview, the vectors and tools READMEs and the changelog point at the new path. The FAQ no longer says the oracle is not a library. pixi.lock is not re-solved in this commit: with a dynamic version, pixi builds the package metadata to lock, which needs the conda environment installed. Run `pixi lock` and commit the result before pushing; the site workflow installs with --locked. pixi run check passes; no computed value, vector, fixture, schema or SQL file changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
40 files changed
+134
-84
No files matched your search
+5
-4
@@ -21,7 +21,7 @@ A fixture is an example file or package under `fixtures/`.
|
||||
| `load.py` | Validates packages, folders or zipped, then loads them into a SQLite database with the tables of `sql/`: `python3 tools/load.py --sqlite lab.db PACKAGE [PACKAGE ...]`. With `--bundle --built-from REV` it builds a new bundle instead, with the full-text search index ([spec/13 §13.5](../spec/13-database.md#135-the-bundle)). |
|
||||
| `site.py` | Builds the website into `_site/`: every file of the repository at its own path, and each Markdown file rendered as a page beside it, with a vocabulary page, folder listings and a search index. It uses only the standard library. `pixi run check` builds it into a temporary folder and checks every link ([deploy/](../deploy/README.md)). |
|
||||
| `stress.py` | Stresses the oracle with random inputs from a seeded generator: `python3 tools/stress.py [--seed N] [--rounds N]`. It checks the invariants the chapters state rather than the vectors' values, such as canonical JSON round trips, corrections keeping the five values consistent with a full recomputation, anchors always resolving to one of the five outcomes, the zip reader never writing outside its folder, and a mutated package always being refused. |
|
||||
| `oracle/` | A compact implementation of every rule. `build.py` uses it to compute expected values, and `check.py` uses it to recompute them. |
|
||||
| [src/paperspec/](../src/paperspec/) | The oracle: a compact implementation of every rule, and the Python package `paperspec`. `build.py` uses it to compute expected values, and `check.py` uses it to recompute them. The programs put `src/` first on the path, so they run this checkout's code with or without pixi. |
|
||||
|
||||
## When to run what
|
||||
|
||||
@@ -48,7 +48,7 @@ Each program prints its options with `--help` and refuses options it does not kn
|
||||
`build.py` rebuilds `fixtures/` and `vectors/` in place unless you pass `--out`.
|
||||
|
||||
Run `pixi run check` before every commit.
|
||||
Run `pixi run build` after you change a fixture source, a rule in `oracle/` or a case in `build.py`, and commit the regenerated files with the change.
|
||||
Run `pixi run build` after you change a fixture source, a rule in `src/paperspec/` or a case in `build.py`, and commit the regenerated files with the change.
|
||||
The check rebuilds into a temporary folder and compares every file byte for byte, so it fails on out-of-date files.
|
||||
|
||||
## About the oracle
|
||||
@@ -59,7 +59,7 @@ The oracle exists so that no expected value in this repository is typed by hand.
|
||||
|
||||
It favors clarity, and leaves the cryptography and schema validation to established packages.
|
||||
Ed25519 (a digital signature scheme) and HKDF come from `cryptography`, and JSON Schema (a format that describes the shape of JSON data) validation from `jsonschema`.
|
||||
The VRF of RFC 9381 (`oracle/vrf.py`) and ML-DSA-65 of FIPS 204 (`oracle/mldsa.py`) are written out in plain Python, slow and clear, and checked against the standards' own test vectors.
|
||||
The VRF of RFC 9381 (`src/paperspec/vrf.py`) and ML-DSA-65 of FIPS 204 (`src/paperspec/mldsa.py`) are written out in plain Python, slow and clear, and checked against the standards' own test vectors.
|
||||
The oracle still refuses any schema that uses a keyword outside the subset the schemas are restricted to.
|
||||
Canonical JSON goes through Python's built-in JSON encoder, and falls back to an exact RFC 8785 serializer for the values that encoder writes differently: floating-point numbers, integers beyond 2^53 and some non-ASCII member names.
|
||||
|
||||
@@ -67,8 +67,9 @@ Hashes are kept incrementally.
|
||||
A node's five values depend only on its own subtree, so after a correction the oracle rehashes only the nodes that changed and their ancestors, and a container's projections are its children's cached projections joined.
|
||||
`check.py` compares the result after every correction with a slow computation that projects each node from its whole subtree.
|
||||
|
||||
Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec` (`pip install .`, or a git pin) and import the oracle, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
|
||||
Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec` (`pip install .`, or a git pin) and import the oracle from it, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
|
||||
The library surface they use is `keys`, `text`, `jcs`, `package`, `db`, `schema` and `rights`, with the schemas, SQL and vectors as package data (`importlib.resources.files("paperspec") / "sql" / "doc.sql"`).
|
||||
Those modules run on Python 3.12 with the standard library alone, except that `schema` needs `paperspec[schema]` (`jsonschema` and `referencing`) and the HKDF and Ed25519 functions of `crypto` need `paperspec[crypto]`; `paperspec[tools]` installs both.
|
||||
The package version comes from git tags through setuptools-scm, and `pixi run -e dev wheel` builds the wheel into `dist/`.
|
||||
When `scienting-core` exists it takes this role behind the same vectors, and the import changes without any computed value changing.
|
||||
Independent implementations follow the [specification](../spec/README.md) and test themselves against the [vectors](../vectors/README.md).
|
||||
Reference in new issue
Block a user