Reorganize paperspec commands and pin dependency versions #2
No files matched your search
@@ -3,6 +3,9 @@ __pycache__/
|
||||
*.db
|
||||
*.db.assets/
|
||||
|
||||
# the package version, written by setuptools-scm when the package is built or installed
|
||||
/src/paperspec/_version.py
|
||||
|
||||
# the website, built by tools/site.py
|
||||
/_site/
|
||||
|
||||
|
||||
+4
-3
@@ -24,7 +24,8 @@ The first release.
|
||||
|
||||
Packaging and lazy imports only; the formats stay at 0.1.0, and no computed value, vector, fixture, schema or SQL file changed.
|
||||
|
||||
- `tools/oracle` installs as the Python package `paperspec` (`pyproject.toml`, setuptools), with `schemas/`, `sql/` and `vectors/` as the data subpackages `paperspec.schemas`, `paperspec.sql` and `paperspec.vectors`, so other repositories import the reference implementation instead of reimplementing it.
|
||||
- The oracle moves from `tools/oracle/` to `src/paperspec/` and installs as the Python package `paperspec` (`pyproject.toml`, setuptools), with `schemas/`, `sql/` and `vectors/` as the data subpackages `paperspec.schemas`, `paperspec.sql` and `paperspec.vectors`, so other repositories import the reference implementation instead of reimplementing it.
|
||||
- The package version comes from git tags through setuptools-scm.
|
||||
- The base package has no dependencies and runs on Python 3.12; the extras `paperspec[schema]` (`jsonschema`, `referencing`), `paperspec[crypto]` (`cryptography`) and `paperspec[tools]` (both) cover `schema` and the HKDF and Ed25519 functions of `crypto`, which now import their packages on first use.
|
||||
- The Python 3.14 and package gate in `tools/oracle/__init__.py` runs only when the repository's own tools import it as `tools.oracle`.
|
||||
- The pixi workspace installs the package editable, so the tools and the package resolve to the same code.
|
||||
- The tools import `paperspec` from `src/`, and their Python 3.14 and package check moves to `tools/__init__.py`, so the package itself checks nothing on import.
|
||||
- The pixi workspace installs the package editable, and a `dev` environment builds the wheel with `pixi run -e dev wheel`.
|
||||
+2
-2
@@ -13,7 +13,7 @@ The schemas, SQL, registries, vectors and fixtures then follow the chapter ([spe
|
||||
2. **Change the schemas, SQL and registries** that the rule shapes.
|
||||
A new refusal gets a rule ID in [registry/rules.json](registry/rules.json).
|
||||
3. **Compute, never type, the expected values.**
|
||||
Change the oracle in [tools/oracle/](tools/oracle/) to apply the rule, and add the case that shows it to [tools/build.py](tools/build.py): a vector, a fixture or a broken package in `fixtures/invalid/`.
|
||||
Change the oracle in [src/paperspec/](src/paperspec/) to apply the rule, and add the case that shows it to [tools/build.py](tools/build.py): a vector, a fixture or a broken package in `fixtures/invalid/`.
|
||||
Then run `pixi run build`, which rewrites `fixtures/` and `vectors/`.
|
||||
4. **Run the checks.**
|
||||
`pixi run check` must end with `All checks pass.`
|
||||
@@ -29,7 +29,7 @@ A rebuild that changes any file under `vectors/` or any hash in `fixtures/` is a
|
||||
## New node kinds and run kinds
|
||||
|
||||
Propose a kind with the vector or fixture that shows it.
|
||||
A node kind needs its row in [chapter 3](spec/03-nodes.md), the node schema, and its entry in the `CHILDREN` table of `tools/oracle/model.py`.
|
||||
A node kind needs its row in [chapter 3](spec/03-nodes.md), the node schema, and its entry in the `CHILDREN` table of `src/paperspec/model.py`.
|
||||
A run kind needs its row in [section 3.5](spec/03-nodes.md#35-inline-runs) and the node schema, and its text in the plain projection ([3.6](spec/03-nodes.md#36-the-plain-projection)) and in the strict and lossy projections ([5.3](spec/05-hashes.md#53-the-strict-projection), [5.5](spec/05-hashes.md#55-the-lossy-projection)).
|
||||
|
||||
## Releasing
|
||||
|
||||
@@ -62,6 +62,7 @@ The address grammar is in [spec/06 §6.1](spec/06-anchors.md#61-addresses), and
|
||||
| [vectors/](vectors/README.md) | Test vectors: inputs and the exact outputs every implementation must compute. |
|
||||
| [fixtures/](fixtures/README.md) | Example packages and records, deliberately broken packages, and the test keys that signed them. |
|
||||
| [tools/](tools/README.md) | The programs that build and check everything above. |
|
||||
| [src/paperspec/](src/paperspec/) | The reference implementation of every rule, installable as the Python package `paperspec`. |
|
||||
| [docs/](docs/README.md) | Guides for people getting to know the project. |
|
||||
| [deploy/](deploy/README.md) | How the website at <https://paperspec.scient.ing> is built and served. |
|
||||
|
||||
|
||||
+2
-2
@@ -127,5 +127,5 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md).
|
||||
## What are the files in `tools/` for?
|
||||
|
||||
`build.py` builds the fixtures and vectors. `check.py` recomputes and checks every one, and `load.py` loads packages into SQLite.
|
||||
The `oracle/` modules compute every expected value, so that none is typed by hand.
|
||||
They are not a library; implementations follow the specification ([tools/README.md](../tools/README.md)).
|
||||
The oracle's modules in `src/paperspec/` compute every expected value, so that none is typed by hand.
|
||||
The other repositories of the project install them as the Python package `paperspec`; independent implementations follow the specification ([tools/README.md](../tools/README.md)).
|
||||
+2
-1
@@ -54,7 +54,8 @@ It appears inside hashed data and test vectors, and it is separate from the repo
|
||||
| [spec/](../spec/README.md) | The specification, in 24 chapters (00 to 23). Normative. |
|
||||
| [schemas/](../schemas/README.md) | 42 JSON (plain-text data format) Schemas for the files and records. |
|
||||
| [registry/](../registry/README.md) | The list of formats, the rule IDs a checker reports, download-stamp rules, and default rights requirements. |
|
||||
| [tools/](../tools/README.md) | Reference programs: `check.py` checks everything, `build.py` rebuilds the fixtures and vectors, `load.py` loads packages into SQLite, and `oracle/` is a compact implementation of every rule. |
|
||||
| [tools/](../tools/README.md) | Reference programs: `check.py` checks everything, `build.py` rebuilds the fixtures and vectors, and `load.py` loads packages into SQLite. |
|
||||
| [src/paperspec/](../src/paperspec/) | The oracle, a compact implementation of every rule, which the other repositories of the project install as the Python package `paperspec`. |
|
||||
| [vectors/](../vectors/README.md) | 25 files of test vectors: inputs and the exact outputs every implementation must compute. |
|
||||
| [fixtures/](../fixtures/README.md) | Five valid example packages, one of them also as a zip, 31 deliberately broken ones that each break one named rule, example records, and the test keys that signed them. All text is synthetic. |
|
||||
| [sql/](../sql/README.md) | Tables for SQLite, Dolt and PostgreSQL, and a SQLite bundle with full-text search. |
|
||||
|
||||
@@ -20,11 +20,24 @@ markdownlint-cli2 = ">=0.23.3"
|
||||
nodejs = ">=22"
|
||||
cryptography = ">=46"
|
||||
|
||||
# The tools import the oracle as `tools.oracle`; other programs install it as `paperspec`. The editable
|
||||
# self-install makes both resolve to the same code.
|
||||
# The reference implementation in src/paperspec, installed editable so that `import paperspec` works in
|
||||
# every pixi environment; the tools put src/ first on the path themselves, so they also run without pixi.
|
||||
[pypi-dependencies]
|
||||
paperspec = { path = ".", editable = true }
|
||||
|
||||
# What building the package needs: pixi run -e dev wheel
|
||||
[feature.dev.dependencies]
|
||||
setuptools = ">=80"
|
||||
setuptools-scm = ">=8"
|
||||
python-build = ">=1.2"
|
||||
|
||||
[feature.dev.tasks]
|
||||
wheel = { cmd = "python -m build --wheel --no-isolation --outdir dist", description = "Build the paperspec wheel into dist/" }
|
||||
|
||||
[environments]
|
||||
default = { solve-group = "default" }
|
||||
dev = { features = ["dev"], solve-group = "default" }
|
||||
|
||||
[tasks]
|
||||
check = { cmd = "python tools/check.py", depends-on = [
|
||||
"lint",
|
||||
|
||||
+14
-6
@@ -1,11 +1,10 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=80", "setuptools-scm>=8"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
requires = ["setuptools>=77"]
|
||||
|
||||
[project]
|
||||
name = "paperspec"
|
||||
version = "0.1.0"
|
||||
dynamic = ["version"]
|
||||
description = "The scienting formats: the reference implementation of every rule, with the JSON Schemas, SQL and test vectors as package data"
|
||||
readme = "README.md"
|
||||
license = "Apache-2.0"
|
||||
@@ -32,13 +31,15 @@ Homepage = "https://paperspec.scient.ing"
|
||||
Repository = "https://git.scient.ing/infra/paperspec"
|
||||
Documentation = "https://paperspec.scient.ing/docs/"
|
||||
|
||||
# `tools/oracle` is the package `paperspec`; the schemas, SQL and vectors travel
|
||||
# with it as data subpackages, so a consumer reads them with importlib.resources.
|
||||
# The code is `src/paperspec`. The schemas, SQL and vectors stay at the repository root, where the
|
||||
# specification links to them and the website serves the schemas at their `$id` URLs, and travel with the
|
||||
# package as data subpackages, so a consumer reads them with importlib.resources. That mapping is why the
|
||||
# packages are listed rather than found.
|
||||
[tool.setuptools]
|
||||
packages = ["paperspec", "paperspec.schemas", "paperspec.sql", "paperspec.vectors"]
|
||||
|
||||
[tool.setuptools.package-dir]
|
||||
"paperspec" = "tools/oracle"
|
||||
"" = "src"
|
||||
"paperspec.schemas" = "schemas"
|
||||
"paperspec.sql" = "sql"
|
||||
"paperspec.vectors" = "vectors"
|
||||
@@ -47,3 +48,10 @@ packages = ["paperspec", "paperspec.schemas", "paperspec.sql", "paperspec.vector
|
||||
"paperspec.schemas" = ["*.json"]
|
||||
"paperspec.sql" = ["*.sql"]
|
||||
"paperspec.vectors" = ["*.jsonl"]
|
||||
|
||||
# The package version comes from git tags (v0.1.0 builds as 0.1.0, later commits as 0.1.1.devN). It is the
|
||||
# version of this code, not of the formats, which CHANGELOG.md and pixi.toml carry.
|
||||
[tool.setuptools_scm]
|
||||
version_file = "src/paperspec/_version.py"
|
||||
version_scheme = "guess-next-dev"
|
||||
local_scheme = "node-and-timestamp"
|
||||
@@ -0,0 +1,14 @@
|
||||
"""The reference implementation of the scienting formats: the computations behind every expected value
|
||||
in this repository; see tools/README.md.
|
||||
|
||||
The modules other programs use (`keys`, `text`, `jcs`, `package`, `db`, `rights`, ...) need only the
|
||||
standard library and Python 3.12.
|
||||
`schema` needs `paperspec[schema]` and the HKDF and Ed25519 functions of `crypto` need `paperspec[crypto]`;
|
||||
both import their packages on first use.
|
||||
The schemas, SQL and vectors travel as package data: `importlib.resources.files("paperspec") / "sql"`.
|
||||
"""
|
||||
|
||||
try:
|
||||
from ._version import __version__
|
||||
except ImportError: # a checkout that was never built or installed, as the repository's tools use it
|
||||
__version__ = "0+unknown"
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
+5
-4
@@ -21,7 +21,7 @@ A fixture is an example file or package under `fixtures/`.
|
||||
| `load.py` | Validates packages, folders or zipped, then loads them into a SQLite database with the tables of `sql/`: `python3 tools/load.py --sqlite lab.db PACKAGE [PACKAGE ...]`. With `--bundle --built-from REV` it builds a new bundle instead, with the full-text search index ([spec/13 §13.5](../spec/13-database.md#135-the-bundle)). |
|
||||
| `site.py` | Builds the website into `_site/`: every file of the repository at its own path, and each Markdown file rendered as a page beside it, with a vocabulary page, folder listings and a search index. It uses only the standard library. `pixi run check` builds it into a temporary folder and checks every link ([deploy/](../deploy/README.md)). |
|
||||
| `stress.py` | Stresses the oracle with random inputs from a seeded generator: `python3 tools/stress.py [--seed N] [--rounds N]`. It checks the invariants the chapters state rather than the vectors' values, such as canonical JSON round trips, corrections keeping the five values consistent with a full recomputation, anchors always resolving to one of the five outcomes, the zip reader never writing outside its folder, and a mutated package always being refused. |
|
||||
| `oracle/` | A compact implementation of every rule. `build.py` uses it to compute expected values, and `check.py` uses it to recompute them. |
|
||||
| [src/paperspec/](../src/paperspec/) | The oracle: a compact implementation of every rule, and the Python package `paperspec`. `build.py` uses it to compute expected values, and `check.py` uses it to recompute them. The programs put `src/` first on the path, so they run this checkout's code with or without pixi. |
|
||||
|
||||
## When to run what
|
||||
|
||||
@@ -48,7 +48,7 @@ Each program prints its options with `--help` and refuses options it does not kn
|
||||
`build.py` rebuilds `fixtures/` and `vectors/` in place unless you pass `--out`.
|
||||
|
||||
Run `pixi run check` before every commit.
|
||||
Run `pixi run build` after you change a fixture source, a rule in `oracle/` or a case in `build.py`, and commit the regenerated files with the change.
|
||||
Run `pixi run build` after you change a fixture source, a rule in `src/paperspec/` or a case in `build.py`, and commit the regenerated files with the change.
|
||||
The check rebuilds into a temporary folder and compares every file byte for byte, so it fails on out-of-date files.
|
||||
|
||||
## About the oracle
|
||||
@@ -59,7 +59,7 @@ The oracle exists so that no expected value in this repository is typed by hand.
|
||||
|
||||
It favors clarity, and leaves the cryptography and schema validation to established packages.
|
||||
Ed25519 (a digital signature scheme) and HKDF come from `cryptography`, and JSON Schema (a format that describes the shape of JSON data) validation from `jsonschema`.
|
||||
The VRF of RFC 9381 (`oracle/vrf.py`) and ML-DSA-65 of FIPS 204 (`oracle/mldsa.py`) are written out in plain Python, slow and clear, and checked against the standards' own test vectors.
|
||||
The VRF of RFC 9381 (`src/paperspec/vrf.py`) and ML-DSA-65 of FIPS 204 (`src/paperspec/mldsa.py`) are written out in plain Python, slow and clear, and checked against the standards' own test vectors.
|
||||
The oracle still refuses any schema that uses a keyword outside the subset the schemas are restricted to.
|
||||
Canonical JSON goes through Python's built-in JSON encoder, and falls back to an exact RFC 8785 serializer for the values that encoder writes differently: floating-point numbers, integers beyond 2^53 and some non-ASCII member names.
|
||||
|
||||
@@ -67,8 +67,9 @@ Hashes are kept incrementally.
|
||||
A node's five values depend only on its own subtree, so after a correction the oracle rehashes only the nodes that changed and their ancestors, and a container's projections are its children's cached projections joined.
|
||||
`check.py` compares the result after every correction with a slow computation that projects each node from its whole subtree.
|
||||
|
||||
Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec` (`pip install .`, or a git pin) and import the oracle, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
|
||||
Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec` (`pip install .`, or a git pin) and import the oracle from it, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
|
||||
The library surface they use is `keys`, `text`, `jcs`, `package`, `db`, `schema` and `rights`, with the schemas, SQL and vectors as package data (`importlib.resources.files("paperspec") / "sql" / "doc.sql"`).
|
||||
Those modules run on Python 3.12 with the standard library alone, except that `schema` needs `paperspec[schema]` (`jsonschema` and `referencing`) and the HKDF and Ed25519 functions of `crypto` need `paperspec[crypto]`; `paperspec[tools]` installs both.
|
||||
The package version comes from git tags through setuptools-scm, and `pixi run -e dev wheel` builds the wheel into `dist/`.
|
||||
When `scienting-core` exists it takes this role behind the same vectors, and the import changes without any computed value changing.
|
||||
Independent implementations follow the [specification](../spec/README.md) and test themselves against the [vectors](../vectors/README.md).
|
||||
@@ -0,0 +1,24 @@
|
||||
"""The repository's own programs; see tools/README.md.
|
||||
|
||||
Importing this package checks that the tools can run here, and puts `src/` first on the path, so that
|
||||
`import paperspec` gets this checkout's reference implementation rather than an installed copy.
|
||||
The tools need Python 3.14 and the packages `cryptography` and `jsonschema`; the `paperspec` package
|
||||
itself does not, and other programs install it on its own.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import os
|
||||
import sys
|
||||
|
||||
if sys.version_info < (3, 14):
|
||||
sys.exit("The paperspec tools need Python 3.14 or later; this is Python %d.%d." % sys.version_info[:2])
|
||||
|
||||
_missing = [m for m in ("cryptography", "jsonschema") if importlib.util.find_spec(m) is None]
|
||||
if _missing:
|
||||
sys.exit("The paperspec tools need %s %s; run them through pixi, or install %s with pip."
|
||||
% ("the package" if len(_missing) == 1 else "the packages", " and ".join(_missing),
|
||||
"it" if len(_missing) == 1 else "them"))
|
||||
|
||||
SRC = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "src")
|
||||
if SRC not in sys.path:
|
||||
sys.path.insert(0, SRC)
|
||||
+13
-12
@@ -20,18 +20,19 @@ from fractions import Fraction
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
|
||||
|
||||
from tools.oracle import align as AL # noqa: E402
|
||||
from tools.oracle import anchors as AN # noqa: E402
|
||||
from tools.oracle import corrections as CO # noqa: E402
|
||||
from tools.oracle import crypto, fingerprint as FP, grade as GR, hashing, jcs, receipts as RC, rights as RI # noqa: E402
|
||||
from tools.oracle import keys as K, log as LG, mldsa, targets as TG, vrf # noqa: E402
|
||||
from tools.oracle.model import LEAVES, MATH_KINDS, RUN_KINDS, VERBATIM_KINDS, Doc # noqa: E402
|
||||
from tools.oracle.notation import fold_math # noqa: E402
|
||||
from tools.oracle.package import ZIP_EXTENSION, history_items, jsonl, line, witness_id, write, write_zip # noqa: E402
|
||||
from tools.oracle.schema import Store # noqa: E402
|
||||
from tools.oracle.text import letter_first, mint_node_id, norm, record_id, sha256_hex # noqa: E402
|
||||
from tools.oracle.tree import Refusal, _flatten, mint_ids, validate_structure # noqa: E402
|
||||
from paperspec import align as AL # noqa: E402
|
||||
from paperspec import anchors as AN # noqa: E402
|
||||
from paperspec import corrections as CO # noqa: E402
|
||||
from paperspec import crypto, fingerprint as FP, grade as GR, hashing, jcs, receipts as RC, rights as RI # noqa: E402
|
||||
from paperspec import keys as K, log as LG, mldsa, targets as TG, vrf # noqa: E402
|
||||
from paperspec.model import LEAVES, MATH_KINDS, RUN_KINDS, VERBATIM_KINDS, Doc # noqa: E402
|
||||
from paperspec.notation import fold_math # noqa: E402
|
||||
from paperspec.package import ZIP_EXTENSION, history_items, jsonl, line, witness_id, write, write_zip # noqa: E402
|
||||
from paperspec.schema import Store # noqa: E402
|
||||
from paperspec.text import letter_first, mint_node_id, norm, record_id, sha256_hex # noqa: E402
|
||||
from paperspec.tree import Refusal, _flatten, mint_ids, validate_structure # noqa: E402
|
||||
|
||||
VERSION = "0.1.0"
|
||||
STORE = "smithlab"
|
||||
@@ -903,7 +904,7 @@ def vectors(out, hale, tiny):
|
||||
("astral", [T("Let "), M("\U0001d465"), T(" vary with "), M("\U0001d466"), T(".")], [4, 7])]:
|
||||
doc, s = sentence_doc(runs)
|
||||
p = hashing.plain(doc, s)
|
||||
from tools.oracle.text import cp_to_utf16
|
||||
from paperspec.text import cp_to_utf16
|
||||
off.append({"id": cid, "runs": runs, "plain": p, "span": span, "exact": p[span[0]:span[1]],
|
||||
"utf16_span": [cp_to_utf16(p, span[0]), cp_to_utf16(p, span[1])], "code_points": len(p),
|
||||
"utf16_units": cp_to_utf16(p, len(p))})
|
||||
|
||||
+15
-14
@@ -20,21 +20,22 @@ from fractions import Fraction
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
|
||||
|
||||
from tools.oracle import align as AL # noqa: E402
|
||||
from tools.oracle import anchors as AN # noqa: E402
|
||||
from tools.oracle import corrections as CO # noqa: E402
|
||||
from tools.oracle import crypto, db, fingerprint as FP, grade as GR, hashing, jcs, receipts as RC, rights as RI # noqa: E402
|
||||
from tools.oracle import package as PK # noqa: E402
|
||||
from tools.oracle import keys as K, log as LG, mldsa, targets as TG, vrf # noqa: E402
|
||||
from paperspec import align as AL # noqa: E402
|
||||
from paperspec import anchors as AN # noqa: E402
|
||||
from paperspec import corrections as CO # noqa: E402
|
||||
from paperspec import crypto, db, fingerprint as FP, grade as GR, hashing, jcs, receipts as RC, rights as RI # noqa: E402
|
||||
from paperspec import package as PK # noqa: E402
|
||||
from paperspec import keys as K, log as LG, mldsa, targets as TG, vrf # noqa: E402
|
||||
import unicodedata # noqa: E402
|
||||
from tools.oracle.model import Doc # noqa: E402
|
||||
from tools.oracle.notation import fold_math # noqa: E402
|
||||
from tools.oracle.schema import Store, validate as sv # noqa: E402
|
||||
from paperspec.model import Doc # noqa: E402
|
||||
from paperspec.notation import fold_math # noqa: E402
|
||||
from paperspec.schema import Store, validate as sv # noqa: E402
|
||||
from jsonschema import Draft202012Validator # noqa: E402
|
||||
from jsonschema.exceptions import SchemaError # noqa: E402
|
||||
from tools.oracle.text import cp_to_utf16, letter_first, mint_node_id, norm, record_id, sha256_hex # noqa: E402
|
||||
from tools.oracle.tree import Refusal # noqa: E402
|
||||
from paperspec.text import cp_to_utf16, letter_first, mint_node_id, norm, record_id, sha256_hex # noqa: E402
|
||||
from paperspec.tree import Refusal # noqa: E402
|
||||
|
||||
FAILS = []
|
||||
# Folders that are not part of the repository: version control, environments and caches.
|
||||
@@ -111,11 +112,11 @@ def check_registry():
|
||||
for rule in js(p("registry", "stamps", "scienting-stamps-1.json"))["rules"]:
|
||||
re.compile(rule["pattern"])
|
||||
rules = {r["id"] for r in js(p("registry", "rules.json"))["rules"]}
|
||||
for f in sorted(os.listdir(p("tools", "oracle"))):
|
||||
for f in sorted(os.listdir(p("src", "paperspec"))):
|
||||
if f.endswith(".py"):
|
||||
for rule in re.findall(r'(?:Refusal|rep\.add)\(\s*"([a-z][a-z0-9.-]+)"', open(p("tools", "oracle", f)).read()):
|
||||
for rule in re.findall(r'(?:Refusal|rep\.add)\(\s*"([a-z][a-z0-9.-]+)"', open(p("src", "paperspec", f)).read()):
|
||||
if not rule.endswith("."):
|
||||
ok("registry", rule in rules, "tools/oracle/%s raises %s, which is not in rules.json" % (f, rule))
|
||||
ok("registry", rule in rules, "src/paperspec/%s raises %s, which is not in rules.json" % (f, rule))
|
||||
for v in ("text-hash", "match-hash", "align-key", "own-hash", "node-hash"):
|
||||
ok("registry", "doc.hash." + v in rules, "doc.hash.%s is not in rules.json" % v)
|
||||
# The character data the hashes depend on is pinned to one Unicode version (spec/01 §1.4).
|
||||
|
||||
+3
-2
@@ -20,9 +20,10 @@ import tempfile
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
|
||||
|
||||
from tools.oracle import db, package # noqa: E402
|
||||
from tools.oracle.schema import Store # noqa: E402
|
||||
from paperspec import db, package # noqa: E402
|
||||
from paperspec.schema import Store # noqa: E402
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
@@ -1,21 +0,0 @@
|
||||
"""The reference computations behind every expected value in this repository; see tools/README.md.
|
||||
|
||||
Imported as `tools.oracle` by the repository's own tools, which need Python 3.14, `cryptography` and
|
||||
`jsonschema` for everything they check; the gate below exits with a message when one is missing.
|
||||
Installed and imported as `paperspec` by other programs, the gate is skipped: the standard-library modules
|
||||
(`keys`, `text`, `jcs`, `package`, `db`, `rights`, ...) run on Python 3.12, and `schema` and `crypto` import
|
||||
their packages on first use, so `paperspec[schema]` and `paperspec[crypto]` are needed only for those two.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import sys
|
||||
|
||||
if __name__ == "tools.oracle":
|
||||
if sys.version_info < (3, 14):
|
||||
sys.exit("The paperspec tools need Python 3.14 or later; this is Python %d.%d." % sys.version_info[:2])
|
||||
|
||||
_missing = [m for m in ("cryptography", "jsonschema") if importlib.util.find_spec(m) is None]
|
||||
if _missing:
|
||||
sys.exit("The paperspec tools need %s %s; run them through pixi, or install %s with pip."
|
||||
% ("the package" if len(_missing) == 1 else "the packages", " and ".join(_missing),
|
||||
"it" if len(_missing) == 1 else "them"))
|
||||
+11
-10
@@ -29,17 +29,18 @@ from math import comb
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
|
||||
|
||||
from tools.oracle import align as AL # noqa: E402
|
||||
from tools.oracle import anchors as AN # noqa: E402
|
||||
from tools.oracle import corrections as CO # noqa: E402
|
||||
from tools.oracle import crypto, fingerprint as FP, hashing, jcs, receipts as RC, rights as RI, targets as TG # noqa: E402
|
||||
from tools.oracle import keys as K, log as LG, mldsa, vrf # noqa: E402
|
||||
from tools.oracle import package as PK # noqa: E402
|
||||
from tools.oracle.model import LABELLED, LEAVES, Doc # noqa: E402
|
||||
from tools.oracle.notation import fold_math # noqa: E402
|
||||
from tools.oracle.text import NODE_ID_RE, RECORD_ID_RE, letter_first, norm, record_id, sha256_hex, is_ws # noqa: E402
|
||||
from tools.oracle.tree import Refusal, validate_structure # noqa: E402
|
||||
from paperspec import align as AL # noqa: E402
|
||||
from paperspec import anchors as AN # noqa: E402
|
||||
from paperspec import corrections as CO # noqa: E402
|
||||
from paperspec import crypto, fingerprint as FP, hashing, jcs, receipts as RC, rights as RI, targets as TG # noqa: E402
|
||||
from paperspec import keys as K, log as LG, mldsa, vrf # noqa: E402
|
||||
from paperspec import package as PK # noqa: E402
|
||||
from paperspec.model import LABELLED, LEAVES, Doc # noqa: E402
|
||||
from paperspec.notation import fold_math # noqa: E402
|
||||
from paperspec.text import NODE_ID_RE, RECORD_ID_RE, letter_first, norm, record_id, sha256_hex, is_ws # noqa: E402
|
||||
from paperspec.tree import Refusal, validate_structure # noqa: E402
|
||||
from tools.build import ROSTER_BY_ID, SCHEMAS, secret, sentence_doc, tiny_package # noqa: E402
|
||||
from tools.check import HALE, STORE, TINY_KEY, reference_values # noqa: E402
|
||||
|
||||
|
||||
+4
-3
@@ -21,10 +21,11 @@ import tempfile
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
sys.path.insert(0, ROOT)
|
||||
import tools # noqa: E402,F401 checks the runtime and puts src/ first on the path, before paperspec is imported
|
||||
|
||||
from tools.oracle import corrections, jcs, package # noqa: E402
|
||||
from tools.oracle.schema import Store # noqa: E402
|
||||
from tools.oracle.tree import Refusal # noqa: E402
|
||||
from paperspec import corrections, jcs, package # noqa: E402
|
||||
from paperspec.schema import Store # noqa: E402
|
||||
from paperspec.tree import Refusal # noqa: E402
|
||||
|
||||
|
||||
def read_proposals(path):
|
||||
|
||||
+2
-2
@@ -85,7 +85,7 @@ python3 tools/check.py --quick
|
||||
```
|
||||
|
||||
It needs Python 3.14 and two packages, `jsonschema` and `cryptography`, and has no option to run one file.
|
||||
It tests the reference implementation in `tools/oracle/` against the vectors.
|
||||
It tests the reference implementation in `src/paperspec/` against the vectors.
|
||||
It does not test yours.
|
||||
|
||||
## What each file tests
|
||||
@@ -136,7 +136,7 @@ The specification, and the vectors with it, are at version 0.1.0 ([0.3](../spec/
|
||||
The specification has 32 formats: 19 are `candidate` and 13 are `provisional` in [registry/formats.json](../registry/formats.json).
|
||||
A candidate format changes only with a recorded reason and a new or changed vector ([22.5](../spec/22-conformance.md#225-stability)).
|
||||
|
||||
A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in `tools/oracle/`.
|
||||
A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in `src/paperspec/`.
|
||||
That code computed the expected outputs, and `pixi run build` regenerates them ([tools/](../tools/README.md)).
|
||||
The vectors hold 311 cases in all, some of them small: `hashes.jsonl` has one document, `alignment.jsonl` one pair of readings, `offsets.jsonl` two cases.
|
||||
|
||||
|
||||
Reference in new issue
Block a user