refactor: move the reference implementation to src/paperspec

The oracle had two import names, `tools.oracle` for the tools and
`paperspec` for other repositories, which could load the same code twice
in one process and needed a gate keyed on the module name. It now lives
in the standard src layout under its one name.

- git mv tools/oracle -> src/paperspec; the 22 rule modules are unchanged.
- The tools import `paperspec`. `tools/__init__.py` now holds the Python
  3.14 and cryptography/jsonschema check and puts src/ first on sys.path,
  so the tools run this checkout's code with or without pixi. The
  package's own __init__ checks nothing and exposes __version__.
- pyproject.toml: version from git tags via setuptools-scm
  (guess-next-dev, node-and-timestamp, src/paperspec/_version.py,
  gitignored). The packages stay listed rather than found because
  schemas/, sql/ and vectors/ stay at the root: the spec links to them and
  the site serves the schemas at their $id URLs.
- pixi.toml: a `dev` feature and environment with setuptools,
  setuptools-scm and python-build, and `pixi run -e dev wheel`.
- check.py scans src/paperspec for rule IDs; README, CONTRIBUTING, the
  FAQ, the overview, the vectors and tools READMEs and the changelog point
  at the new path. The FAQ no longer says the oracle is not a library.

pixi.lock is not re-solved in this commit: with a dynamic version, pixi
builds the package metadata to lock, which needs the conda environment
installed. Run `pixi lock` and commit the result before pushing; the site
workflow installs with --locked.

pixi run check passes; no computed value, vector, fixture, schema or SQL
file changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-15 09:00:40 -04:00
1 parent 2b4467d315
commit a4eecb4d5b
40 files changed
+134 -84

No files matched your search

+3
View File
@@ -3,6 +3,9 @@ __pycache__/
*.db
*.db.assets/
# the package version, written by setuptools-scm when the package is built or installed
/src/paperspec/_version.py
# the website, built by tools/site.py
/_site/
+4 -3
View File
@@ -24,7 +24,8 @@ The first release.
Packaging and lazy imports only; the formats stay at 0.1.0, and no computed value, vector, fixture, schema or SQL file changed.
- `tools/oracle` installs as the Python package `paperspec` (`pyproject.toml`, setuptools), with `schemas/`, `sql/` and `vectors/` as the data subpackages `paperspec.schemas`, `paperspec.sql` and `paperspec.vectors`, so other repositories import the reference implementation instead of reimplementing it.
- The oracle moves from `tools/oracle/` to `src/paperspec/` and installs as the Python package `paperspec` (`pyproject.toml`, setuptools), with `schemas/`, `sql/` and `vectors/` as the data subpackages `paperspec.schemas`, `paperspec.sql` and `paperspec.vectors`, so other repositories import the reference implementation instead of reimplementing it.
- The package version comes from git tags through setuptools-scm.
- The base package has no dependencies and runs on Python 3.12; the extras `paperspec[schema]` (`jsonschema`, `referencing`), `paperspec[crypto]` (`cryptography`) and `paperspec[tools]` (both) cover `schema` and the HKDF and Ed25519 functions of `crypto`, which now import their packages on first use.
- The Python 3.14 and package gate in `tools/oracle/__init__.py` runs only when the repository's own tools import it as `tools.oracle`.
- The pixi workspace installs the package editable, so the tools and the package resolve to the same code.
- The tools import `paperspec` from `src/`, and their Python 3.14 and package check moves to `tools/__init__.py`, so the package itself checks nothing on import.
- The pixi workspace installs the package editable, and a `dev` environment builds the wheel with `pixi run -e dev wheel`.
+2 -2
View File
@@ -13,7 +13,7 @@ The schemas, SQL, registries, vectors and fixtures then follow the chapter ([spe
2. **Change the schemas, SQL and registries** that the rule shapes.
A new refusal gets a rule ID in [registry/rules.json](registry/rules.json).
3. **Compute, never type, the expected values.**
Change the oracle in [tools/oracle/](tools/oracle/) to apply the rule, and add the case that shows it to [tools/build.py](tools/build.py): a vector, a fixture or a broken package in `fixtures/invalid/`.
Change the oracle in [src/paperspec/](src/paperspec/) to apply the rule, and add the case that shows it to [tools/build.py](tools/build.py): a vector, a fixture or a broken package in `fixtures/invalid/`.
Then run `pixi run build`, which rewrites `fixtures/` and `vectors/`.
4. **Run the checks.**
`pixi run check` must end with `All checks pass.`
@@ -29,7 +29,7 @@ A rebuild that changes any file under `vectors/` or any hash in `fixtures/` is a
## New node kinds and run kinds
Propose a kind with the vector or fixture that shows it.
A node kind needs its row in [chapter 3](spec/03-nodes.md), the node schema, and its entry in the `CHILDREN` table of `tools/oracle/model.py`.
A node kind needs its row in [chapter 3](spec/03-nodes.md), the node schema, and its entry in the `CHILDREN` table of `src/paperspec/model.py`.
A run kind needs its row in [section 3.5](spec/03-nodes.md#35-inline-runs) and the node schema, and its text in the plain projection ([3.6](spec/03-nodes.md#36-the-plain-projection)) and in the strict and lossy projections ([5.3](spec/05-hashes.md#53-the-strict-projection), [5.5](spec/05-hashes.md#55-the-lossy-projection)).
## Releasing
+1
View File
@@ -62,6 +62,7 @@ The address grammar is in [spec/06 §6.1](spec/06-anchors.md#61-addresses), and
| [vectors/](vectors/README.md) | Test vectors: inputs and the exact outputs every implementation must compute. |
| [fixtures/](fixtures/README.md) | Example packages and records, deliberately broken packages, and the test keys that signed them. |
| [tools/](tools/README.md) | The programs that build and check everything above. |
| [src/paperspec/](src/paperspec/) | The reference implementation of every rule, installable as the Python package `paperspec`. |
| [docs/](docs/README.md) | Guides for people getting to know the project. |
| [deploy/](deploy/README.md) | How the website at <https://paperspec.scient.ing> is built and served. |
+2 -2
View File
@@ -127,5 +127,5 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md).
## What are the files in `tools/` for?
`build.py` builds the fixtures and vectors. `check.py` recomputes and checks every one, and `load.py` loads packages into SQLite.
The `oracle/` modules compute every expected value, so that none is typed by hand.
They are not a library; implementations follow the specification ([tools/README.md](../tools/README.md)).
The oracle's modules in `src/paperspec/` compute every expected value, so that none is typed by hand.
The other repositories of the project install them as the Python package `paperspec`; independent implementations follow the specification ([tools/README.md](../tools/README.md)).
+2 -1
View File
@@ -54,7 +54,8 @@ It appears inside hashed data and test vectors, and it is separate from the repo
| [spec/](../spec/README.md) | The specification, in 24 chapters (00 to 23). Normative. |
| [schemas/](../schemas/README.md) | 42 JSON (plain-text data format) Schemas for the files and records. |
| [registry/](../registry/README.md) | The list of formats, the rule IDs a checker reports, download-stamp rules, and default rights requirements. |
| [tools/](../tools/README.md) | Reference programs: `check.py` checks everything, `build.py` rebuilds the fixtures and vectors, `load.py` loads packages into SQLite, and `oracle/` is a compact implementation of every rule. |
| [tools/](../tools/README.md) | Reference programs: `check.py` checks everything, `build.py` rebuilds the fixtures and vectors, and `load.py` loads packages into SQLite. |
| [src/paperspec/](../src/paperspec/) | The oracle, a compact implementation of every rule, which the other repositories of the project install as the Python package `paperspec`. |
| [vectors/](../vectors/README.md) | 25 files of test vectors: inputs and the exact outputs every implementation must compute. |
| [fixtures/](../fixtures/README.md) | Five valid example packages, one of them also as a zip, 31 deliberately broken ones that each break one named rule, example records, and the test keys that signed them. All text is synthetic. |
| [sql/](../sql/README.md) | Tables for SQLite, Dolt and PostgreSQL, and a SQLite bundle with full-text search. |
+15 -2
View File
@@ -20,11 +20,24 @@ markdownlint-cli2 = ">=0.23.3"
nodejs = ">=22"
cryptography = ">=46"
# The tools import the oracle as `tools.oracle`; other programs install it as `paperspec`. The editable
# self-install makes both resolve to the same code.
# The reference implementation in src/paperspec, installed editable so that `import paperspec` works in
# every pixi environment; the tools put src/ first on the path themselves, so they also run without pixi.
[pypi-dependencies]
paperspec = { path = ".", editable = true }
# What building the package needs: pixi run -e dev wheel
[feature.dev.dependencies]
setuptools = ">=80"
setuptools-scm = ">=8"
python-build = ">=1.2"
[feature.dev.tasks]
wheel = { cmd = "python -m build --wheel --no-isolation --outdir dist", description = "Build the paperspec wheel into dist/" }
[environments]
default = { solve-group = "default" }
dev = { features = ["dev"], solve-group = "default" }
[tasks]
check = { cmd = "python tools/check.py", depends-on = [
"lint",
+14 -6
View File
@@ -1,11 +1,10 @@
[build-system]
requires = ["setuptools>=80", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"
requires = ["setuptools>=77"]
[project]
name = "paperspec"
version = "0.1.0"
dynamic = ["version"]
description = "The scienting formats: the reference implementation of every rule, with the JSON Schemas, SQL and test vectors as package data"
readme = "README.md"
license = "Apache-2.0"
@@ -32,13 +31,15 @@ Homepage = "https://paperspec.scient.ing"
Repository = "https://git.scient.ing/infra/paperspec"
Documentation = "https://paperspec.scient.ing/docs/"
# `tools/oracle` is the package `paperspec`; the schemas, SQL and vectors travel
# with it as data subpackages, so a consumer reads them with importlib.resources.
# The code is `src/paperspec`. The schemas, SQL and vectors stay at the repository root, where the
# specification links to them and the website serves the schemas at their `$id` URLs, and travel with the
# package as data subpackages, so a consumer reads them with importlib.resources. That mapping is why the
# packages are listed rather than found.
[tool.setuptools]
packages = ["paperspec", "paperspec.schemas", "paperspec.sql", "paperspec.vectors"]
[tool.setuptools.package-dir]
"paperspec" = "tools/oracle"
"" = "src"
"paperspec.schemas" = "schemas"
"paperspec.sql" = "sql"
"paperspec.vectors" = "vectors"
@@ -47,3 +48,10 @@ packages = ["paperspec", "paperspec.schemas", "paperspec.sql", "paperspec.vector
"paperspec.schemas" = ["*.json"]
"paperspec.sql" = ["*.sql"]
"paperspec.vectors" = ["*.jsonl"]
# The package version comes from git tags (v0.1.0 builds as 0.1.0, later commits as 0.1.1.devN). It is the
# version of this code, not of the formats, which CHANGELOG.md and pixi.toml carry.
[tool.setuptools_scm]
version_file = "src/paperspec/_version.py"
version_scheme = "guess-next-dev"
local_scheme = "node-and-timestamp"
+14
View File
@@ -0,0 +1,14 @@
"""The reference implementation of the scienting formats: the computations behind every expected value
in this repository; see tools/README.md.
The modules other programs use (`keys`, `text`, `jcs`, `package`, `db`, `rights`, ...) need only the
standard library and Python 3.12.
`schema` needs `paperspec[schema]` and the HKDF and Ed25519 functions of `crypto` need `paperspec[crypto]`;
both import their packages on first use.
The schemas, SQL and vectors travel as package data: `importlib.resources.files("paperspec") / "sql"`.
"""
try:
from ._version import __version__
except ImportError: # a checkout that was never built or installed, as the repository's tools use it
__version__ = "0+unknown"
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
+5 -4
View File
@@ -21,7 +21,7 @@ A fixture is an example file or package under `fixtures/`.
| `load.py` | Validates packages, folders or zipped, then loads them into a SQLite database with the tables of `sql/`: `python3 tools/load.py --sqlite lab.db PACKAGE [PACKAGE ...]`. With `--bundle --built-from REV` it builds a new bundle instead, with the full-text search index ([spec/13 §13.5](../spec/13-database.md#135-the-bundle)). |
| `site.py` | Builds the website into `_site/`: every file of the repository at its own path, and each Markdown file rendered as a page beside it, with a vocabulary page, folder listings and a search index. It uses only the standard library. `pixi run check` builds it into a temporary folder and checks every link ([deploy/](../deploy/README.md)). |
| `stress.py` | Stresses the oracle with random inputs from a seeded generator: `python3 tools/stress.py [--seed N] [--rounds N]`. It checks the invariants the chapters state rather than the vectors' values, such as canonical JSON round trips, corrections keeping the five values consistent with a full recomputation, anchors always resolving to one of the five outcomes, the zip reader never writing outside its folder, and a mutated package always being refused. |
| `oracle/` | A compact implementation of every rule. `build.py` uses it to compute expected values, and `check.py` uses it to recompute them. |
| [src/paperspec/](../src/paperspec/) | The oracle: a compact implementation of every rule, and the Python package `paperspec`. `build.py` uses it to compute expected values, and `check.py` uses it to recompute them. The programs put `src/` first on the path, so they run this checkout's code with or without pixi. |
## When to run what
@@ -48,7 +48,7 @@ Each program prints its options with `--help` and refuses options it does not kn
`build.py` rebuilds `fixtures/` and `vectors/` in place unless you pass `--out`.
Run `pixi run check` before every commit.
Run `pixi run build` after you change a fixture source, a rule in `oracle/` or a case in `build.py`, and commit the regenerated files with the change.
Run `pixi run build` after you change a fixture source, a rule in `src/paperspec/` or a case in `build.py`, and commit the regenerated files with the change.
The check rebuilds into a temporary folder and compares every file byte for byte, so it fails on out-of-date files.
## About the oracle
@@ -59,7 +59,7 @@ The oracle exists so that no expected value in this repository is typed by hand.
It favors clarity, and leaves the cryptography and schema validation to established packages.
Ed25519 (a digital signature scheme) and HKDF come from `cryptography`, and JSON Schema (a format that describes the shape of JSON data) validation from `jsonschema`.
The VRF of RFC 9381 (`oracle/vrf.py`) and ML-DSA-65 of FIPS 204 (`oracle/mldsa.py`) are written out in plain Python, slow and clear, and checked against the standards' own test vectors.
The VRF of RFC 9381 (`src/paperspec/vrf.py`) and ML-DSA-65 of FIPS 204 (`src/paperspec/mldsa.py`) are written out in plain Python, slow and clear, and checked against the standards' own test vectors.
The oracle still refuses any schema that uses a keyword outside the subset the schemas are restricted to.
Canonical JSON goes through Python's built-in JSON encoder, and falls back to an exact RFC 8785 serializer for the values that encoder writes differently: floating-point numbers, integers beyond 2^53 and some non-ASCII member names.
@@ -67,8 +67,9 @@ Hashes are kept incrementally.
A node's five values depend only on its own subtree, so after a correction the oracle rehashes only the nodes that changed and their ancestors, and a container's projections are its children's cached projections joined.
`check.py` compares the result after every correction with a slow computation that projects each node from its whole subtree.
Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec` (`pip install .`, or a git pin) and import the oracle, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec` (`pip install .`, or a git pin) and import the oracle from it, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
The library surface they use is `keys`, `text`, `jcs`, `package`, `db`, `schema` and `rights`, with the schemas, SQL and vectors as package data (`importlib.resources.files("paperspec") / "sql" / "doc.sql"`).
Those modules run on Python 3.12 with the standard library alone, except that `schema` needs `paperspec[schema]` (`jsonschema` and `referencing`) and the HKDF and Ed25519 functions of `crypto` need `paperspec[crypto]`; `paperspec[tools]` installs both.
The package version comes from git tags through setuptools-scm, and `pixi run -e dev wheel` builds the wheel into `dist/`.
When `scienting-core` exists it takes this role behind the same vectors, and the import changes without any computed value changing.
Independent implementations follow the [specification](../spec/README.md) and test themselves against the [vectors](../vectors/README.md).
+24
View File
@@ -0,0 +1,24 @@
"""The repository's own programs; see tools/README.md.
Importing this package checks that the tools can run here, and puts `src/` first on the path, so that
`import paperspec` gets this checkout's reference implementation rather than an installed copy.
The tools need Python 3.14 and the packages `cryptography` and `jsonschema`; the `paperspec` package
itself does not, and other programs install it on its own.
"""
import importlib.util
import os
import sys
if sys.version_info < (3, 14):
sys.exit("The paperspec tools need Python 3.14 or later; this is Python %d.%d." % sys.version_info[:2])
_missing = [m for m in ("cryptography", "jsonschema") if importlib.util.find_spec(m) is None]
if _missing:
sys.exit("The paperspec tools need %s %s; run them through pixi, or install %s with pip."
% ("the package" if len(_missing) == 1 else "the packages", " and ".join(_missing),
"it" if len(_missing) == 1 else "them"))
SRC = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "src")
if SRC not in sys.path:
sys.path.insert(0, SRC)
+13 -12
View File
@@ -20,18 +20,19 @@ from fractions import Fraction
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
from tools.oracle import align as AL # noqa: E402
from tools.oracle import anchors as AN # noqa: E402
from tools.oracle import corrections as CO # noqa: E402
from tools.oracle import crypto, fingerprint as FP, grade as GR, hashing, jcs, receipts as RC, rights as RI # noqa: E402
from tools.oracle import keys as K, log as LG, mldsa, targets as TG, vrf # noqa: E402
from tools.oracle.model import LEAVES, MATH_KINDS, RUN_KINDS, VERBATIM_KINDS, Doc # noqa: E402
from tools.oracle.notation import fold_math # noqa: E402
from tools.oracle.package import ZIP_EXTENSION, history_items, jsonl, line, witness_id, write, write_zip # noqa: E402
from tools.oracle.schema import Store # noqa: E402
from tools.oracle.text import letter_first, mint_node_id, norm, record_id, sha256_hex # noqa: E402
from tools.oracle.tree import Refusal, _flatten, mint_ids, validate_structure # noqa: E402
from paperspec import align as AL # noqa: E402
from paperspec import anchors as AN # noqa: E402
from paperspec import corrections as CO # noqa: E402
from paperspec import crypto, fingerprint as FP, grade as GR, hashing, jcs, receipts as RC, rights as RI # noqa: E402
from paperspec import keys as K, log as LG, mldsa, targets as TG, vrf # noqa: E402
from paperspec.model import LEAVES, MATH_KINDS, RUN_KINDS, VERBATIM_KINDS, Doc # noqa: E402
from paperspec.notation import fold_math # noqa: E402
from paperspec.package import ZIP_EXTENSION, history_items, jsonl, line, witness_id, write, write_zip # noqa: E402
from paperspec.schema import Store # noqa: E402
from paperspec.text import letter_first, mint_node_id, norm, record_id, sha256_hex # noqa: E402
from paperspec.tree import Refusal, _flatten, mint_ids, validate_structure # noqa: E402
VERSION = "0.1.0"
STORE = "smithlab"
@@ -903,7 +904,7 @@ def vectors(out, hale, tiny):
("astral", [T("Let "), M("\U0001d465"), T(" vary with "), M("\U0001d466"), T(".")], [4, 7])]:
doc, s = sentence_doc(runs)
p = hashing.plain(doc, s)
from tools.oracle.text import cp_to_utf16
from paperspec.text import cp_to_utf16
off.append({"id": cid, "runs": runs, "plain": p, "span": span, "exact": p[span[0]:span[1]],
"utf16_span": [cp_to_utf16(p, span[0]), cp_to_utf16(p, span[1])], "code_points": len(p),
"utf16_units": cp_to_utf16(p, len(p))})
+15 -14
View File
@@ -20,21 +20,22 @@ from fractions import Fraction
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
from tools.oracle import align as AL # noqa: E402
from tools.oracle import anchors as AN # noqa: E402
from tools.oracle import corrections as CO # noqa: E402
from tools.oracle import crypto, db, fingerprint as FP, grade as GR, hashing, jcs, receipts as RC, rights as RI # noqa: E402
from tools.oracle import package as PK # noqa: E402
from tools.oracle import keys as K, log as LG, mldsa, targets as TG, vrf # noqa: E402
from paperspec import align as AL # noqa: E402
from paperspec import anchors as AN # noqa: E402
from paperspec import corrections as CO # noqa: E402
from paperspec import crypto, db, fingerprint as FP, grade as GR, hashing, jcs, receipts as RC, rights as RI # noqa: E402
from paperspec import package as PK # noqa: E402
from paperspec import keys as K, log as LG, mldsa, targets as TG, vrf # noqa: E402
import unicodedata # noqa: E402
from tools.oracle.model import Doc # noqa: E402
from tools.oracle.notation import fold_math # noqa: E402
from tools.oracle.schema import Store, validate as sv # noqa: E402
from paperspec.model import Doc # noqa: E402
from paperspec.notation import fold_math # noqa: E402
from paperspec.schema import Store, validate as sv # noqa: E402
from jsonschema import Draft202012Validator # noqa: E402
from jsonschema.exceptions import SchemaError # noqa: E402
from tools.oracle.text import cp_to_utf16, letter_first, mint_node_id, norm, record_id, sha256_hex # noqa: E402
from tools.oracle.tree import Refusal # noqa: E402
from paperspec.text import cp_to_utf16, letter_first, mint_node_id, norm, record_id, sha256_hex # noqa: E402
from paperspec.tree import Refusal # noqa: E402
FAILS = []
# Folders that are not part of the repository: version control, environments and caches.
@@ -111,11 +112,11 @@ def check_registry():
for rule in js(p("registry", "stamps", "scienting-stamps-1.json"))["rules"]:
re.compile(rule["pattern"])
rules = {r["id"] for r in js(p("registry", "rules.json"))["rules"]}
for f in sorted(os.listdir(p("tools", "oracle"))):
for f in sorted(os.listdir(p("src", "paperspec"))):
if f.endswith(".py"):
for rule in re.findall(r'(?:Refusal|rep\.add)\(\s*"([a-z][a-z0-9.-]+)"', open(p("tools", "oracle", f)).read()):
for rule in re.findall(r'(?:Refusal|rep\.add)\(\s*"([a-z][a-z0-9.-]+)"', open(p("src", "paperspec", f)).read()):
if not rule.endswith("."):
ok("registry", rule in rules, "tools/oracle/%s raises %s, which is not in rules.json" % (f, rule))
ok("registry", rule in rules, "src/paperspec/%s raises %s, which is not in rules.json" % (f, rule))
for v in ("text-hash", "match-hash", "align-key", "own-hash", "node-hash"):
ok("registry", "doc.hash." + v in rules, "doc.hash.%s is not in rules.json" % v)
# The character data the hashes depend on is pinned to one Unicode version (spec/01 §1.4).
+3 -2
View File
@@ -20,9 +20,10 @@ import tempfile
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
from tools.oracle import db, package # noqa: E402
from tools.oracle.schema import Store # noqa: E402
from paperspec import db, package # noqa: E402
from paperspec.schema import Store # noqa: E402
def main():
-21
View File
@@ -1,21 +0,0 @@
"""The reference computations behind every expected value in this repository; see tools/README.md.
Imported as `tools.oracle` by the repository's own tools, which need Python 3.14, `cryptography` and
`jsonschema` for everything they check; the gate below exits with a message when one is missing.
Installed and imported as `paperspec` by other programs, the gate is skipped: the standard-library modules
(`keys`, `text`, `jcs`, `package`, `db`, `rights`, ...) run on Python 3.12, and `schema` and `crypto` import
their packages on first use, so `paperspec[schema]` and `paperspec[crypto]` are needed only for those two.
"""
import importlib.util
import sys
if __name__ == "tools.oracle":
if sys.version_info < (3, 14):
sys.exit("The paperspec tools need Python 3.14 or later; this is Python %d.%d." % sys.version_info[:2])
_missing = [m for m in ("cryptography", "jsonschema") if importlib.util.find_spec(m) is None]
if _missing:
sys.exit("The paperspec tools need %s %s; run them through pixi, or install %s with pip."
% ("the package" if len(_missing) == 1 else "the packages", " and ".join(_missing),
"it" if len(_missing) == 1 else "them"))
+11 -10
View File
@@ -29,17 +29,18 @@ from math import comb
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
from tools.oracle import align as AL # noqa: E402
from tools.oracle import anchors as AN # noqa: E402
from tools.oracle import corrections as CO # noqa: E402
from tools.oracle import crypto, fingerprint as FP, hashing, jcs, receipts as RC, rights as RI, targets as TG # noqa: E402
from tools.oracle import keys as K, log as LG, mldsa, vrf # noqa: E402
from tools.oracle import package as PK # noqa: E402
from tools.oracle.model import LABELLED, LEAVES, Doc # noqa: E402
from tools.oracle.notation import fold_math # noqa: E402
from tools.oracle.text import NODE_ID_RE, RECORD_ID_RE, letter_first, norm, record_id, sha256_hex, is_ws # noqa: E402
from tools.oracle.tree import Refusal, validate_structure # noqa: E402
from paperspec import align as AL # noqa: E402
from paperspec import anchors as AN # noqa: E402
from paperspec import corrections as CO # noqa: E402
from paperspec import crypto, fingerprint as FP, hashing, jcs, receipts as RC, rights as RI, targets as TG # noqa: E402
from paperspec import keys as K, log as LG, mldsa, vrf # noqa: E402
from paperspec import package as PK # noqa: E402
from paperspec.model import LABELLED, LEAVES, Doc # noqa: E402
from paperspec.notation import fold_math # noqa: E402
from paperspec.text import NODE_ID_RE, RECORD_ID_RE, letter_first, norm, record_id, sha256_hex, is_ws # noqa: E402
from paperspec.tree import Refusal, validate_structure # noqa: E402
from tools.build import ROSTER_BY_ID, SCHEMAS, secret, sentence_doc, tiny_package # noqa: E402
from tools.check import HALE, STORE, TINY_KEY, reference_values # noqa: E402
+4 -3
View File
@@ -21,10 +21,11 @@ import tempfile
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
from tools.oracle import corrections, jcs, package # noqa: E402
from tools.oracle.schema import Store # noqa: E402
from tools.oracle.tree import Refusal # noqa: E402
from paperspec import corrections, jcs, package # noqa: E402
from paperspec.schema import Store # noqa: E402
from paperspec.tree import Refusal # noqa: E402
def read_proposals(path):
+2 -2
View File
@@ -85,7 +85,7 @@ python3 tools/check.py --quick
```
It needs Python 3.14 and two packages, `jsonschema` and `cryptography`, and has no option to run one file.
It tests the reference implementation in `tools/oracle/` against the vectors.
It tests the reference implementation in `src/paperspec/` against the vectors.
It does not test yours.
## What each file tests
@@ -136,7 +136,7 @@ The specification, and the vectors with it, are at version 0.1.0 ([0.3](../spec/
The specification has 32 formats: 19 are `candidate` and 13 are `provisional` in [registry/formats.json](../registry/formats.json).
A candidate format changes only with a recorded reason and a new or changed vector ([22.5](../spec/22-conformance.md#225-stability)).
A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in `tools/oracle/`.
A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in `src/paperspec/`.
That code computed the expected outputs, and `pixi run build` regenerates them ([tools/](../tools/README.md)).
The vectors hold 311 cases in all, some of them small: `hashes.jsonl` has one document, `alignment.jsonl` one pair of readings, `offsets.jsonl` two cases.