feat: install tools/oracle as the paperspec package

Other repositories (paperstack, then paperpack) import the reference
implementation instead of reimplementing the specification; scienting-core
will take the role later behind the same vectors.

- pyproject.toml: distribution and import name `paperspec`, version 0.1.0,
  Python >= 3.12, no base dependencies; `tools/oracle` is the package, and
  `schemas/`, `sql/` and `vectors/` ship as the data subpackages
  `paperspec.schemas`, `paperspec.sql` and `paperspec.vectors`. Extras
  `schema`, `crypto` and `tools`.
- tools/oracle/__init__.py: the Python 3.14 and cryptography/jsonschema gate
  runs only when imported as `tools.oracle` by the repository's own tools.
- crypto.py imports `cryptography` on first use of HKDF and Ed25519, with an
  ImportError naming `paperspec[crypto]`; schema.py imports `jsonschema` and
  `referencing` in `Store.validator` and `Store.registry`. package.py, db.py
  and receipts.py then import on a bare interpreter.
- pixi.toml installs the package editable (pixi.lock re-solved; no conda pin
  changed), so the tools and the package resolve to the same code.
- tools/README.md names the library surface and the extras; CHANGELOG.md
  records the packaging change under 0.1.0, since check.py requires the
  changelog's first heading to equal the format version.

No computed value, vector, fixture, schema or SQL file changed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

CHANGELOG.md

@@ -19,3 +19,12 @@ The first release.
 - Guides for newcomers: an overview, the concepts, a walkthrough of the example paper, runnable examples, the rationale, a FAQ and a glossary.
 - Tools that build and check everything, validate packages and zipped packages and run proposed corrections through the gate, load packages into SQLite or a bundle, and stress the reference implementation with random inputs.
 - A pixi workspace pinned to Python 3.14, with tasks for checking, building, linting, validating and loading.
+
+### 0.1.1
+
+Packaging and lazy imports only; the formats stay at 0.1.0, and no computed value, vector, fixture, schema or SQL file changed.
+
+- `tools/oracle` installs as the Python package `paperspec` (`pyproject.toml`, setuptools), with `schemas/`, `sql/` and `vectors/` as the data subpackages `paperspec.schemas`, `paperspec.sql` and `paperspec.vectors`, so other repositories import the reference implementation instead of reimplementing it.
+- The base package has no dependencies and runs on Python 3.12; the extras `paperspec[schema]` (`jsonschema`, `referencing`), `paperspec[crypto]` (`cryptography`) and `paperspec[tools]` (both) cover `schema` and the HKDF and Ed25519 functions of `crypto`, which now import their packages on first use.
+- The Python 3.14 and package gate in `tools/oracle/__init__.py` runs only when the repository's own tools import it as `tools.oracle`.
+- The pixi workspace installs the package editable, so the tools and the package resolve to the same code.

pixi.toml

@@ -20,6 +20,11 @@ markdownlint-cli2 = ">=0.23.3"
 nodejs = ">=22"
 cryptography = ">=46"
 
+# The tools import the oracle as `tools.oracle`; other programs install it as `paperspec`. The editable
+# self-install makes both resolve to the same code.
+[pypi-dependencies]
+paperspec = { path = ".", editable = true }
+
 [tasks]
 check = { cmd = "python tools/check.py", depends-on = [
     "lint",

pyproject.toml

@@ -0,0 +1,49 @@
+[build-system]
+build-backend = "setuptools.build_meta"
+
+requires = ["setuptools>=77"]
+
+[project]
+name = "paperspec"
+version = "0.1.0"
+description = "The scienting formats: the reference implementation of every rule, with the JSON Schemas, SQL and test vectors as package data"
+readme = "README.md"
+license = "Apache-2.0"
+license-files = ["LICENSE.md"]
+authors = [{ name = "Scientific Computing Studio", email = "maintainer@scient.ing" }]
+requires-python = ">=3.12"
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Science/Research",
+    "Programming Language :: Python :: 3 :: Only",
+    "Topic :: Scientific/Engineering",
+]
+# The base package is standard library only. `schema.py` needs jsonschema and
+# `crypto.py` needs cryptography; the repository's own tools need both.
+dependencies = []
+
+[project.optional-dependencies]
+schema = ["jsonschema>=4.26", "referencing>=0.35"]
+crypto = ["cryptography>=46"]
+tools = ["paperspec[crypto,schema]"]
+
+[project.urls]
+Homepage = "https://paperspec.scient.ing"
+Repository = "https://git.scient.ing/infra/paperspec"
+Documentation = "https://paperspec.scient.ing/docs/"
+
+# `tools/oracle` is the package `paperspec`; the schemas, SQL and vectors travel
+# with it as data subpackages, so a consumer reads them with importlib.resources.
+[tool.setuptools]
+packages = ["paperspec", "paperspec.schemas", "paperspec.sql", "paperspec.vectors"]
+
+[tool.setuptools.package-dir]
+"paperspec" = "tools/oracle"
+"paperspec.schemas" = "schemas"
+"paperspec.sql" = "sql"
+"paperspec.vectors" = "vectors"
+
+[tool.setuptools.package-data]
+"paperspec.schemas" = ["*.json"]
+"paperspec.sql" = ["*.sql"]
+"paperspec.vectors" = ["*.jsonl"]

tools/README.md

@@ -67,5 +67,8 @@ Hashes are kept incrementally.
 A node's five values depend only on its own subtree, so after a correction the oracle rehashes only the nodes that changed and their ancestors, and a container's projections are its children's cached projections joined.
 `check.py` compares the result after every correction with a slow computation that projects each node from its whole subtree.
 
-Do not build on it as a library.
+Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec` (`pip install .`, or a git pin) and import the oracle, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
-Implementations follow the [specification](../spec/README.md) and test themselves against the [vectors](../vectors/README.md).
+The library surface they use is `keys`, `text`, `jcs`, `package`, `db`, `schema` and `rights`, with the schemas, SQL and vectors as package data (`importlib.resources.files("paperspec") / "sql" / "doc.sql"`).
+Those modules run on Python 3.12 with the standard library alone, except that `schema` needs `paperspec[schema]` (`jsonschema` and `referencing`) and the HKDF and Ed25519 functions of `crypto` need `paperspec[crypto]`; `paperspec[tools]` installs both.
+When `scienting-core` exists it takes this role behind the same vectors, and the import changes without any computed value changing.
+Independent implementations follow the [specification](../spec/README.md) and test themselves against the [vectors](../vectors/README.md).

tools/oracle/__init__.py

@@ -1,13 +1,21 @@
-"""The reference computations behind every expected value in this repository; see tools/README.md."""
+"""The reference computations behind every expected value in this repository; see tools/README.md.
+
+Imported as `tools.oracle` by the repository's own tools, which need Python 3.14, `cryptography` and
+`jsonschema` for everything they check; the gate below exits with a message when one is missing.
+Installed and imported as `paperspec` by other programs, the gate is skipped: the standard-library modules
+(`keys`, `text`, `jcs`, `package`, `db`, `rights`, ...) run on Python 3.12, and `schema` and `crypto` import
+their packages on first use, so `paperspec[schema]` and `paperspec[crypto]` are needed only for those two.
+"""
 
 import importlib.util
 import sys
 
-if sys.version_info < (3, 14):
+if __name__ == "tools.oracle":
+    if sys.version_info < (3, 14):
         sys.exit("The paperspec tools need Python 3.14 or later; this is Python %d.%d." % sys.version_info[:2])
 
-_missing = [m for m in ("cryptography", "jsonschema") if importlib.util.find_spec(m) is None]
+    _missing = [m for m in ("cryptography", "jsonschema") if importlib.util.find_spec(m) is None]
-if _missing:
+    if _missing:
         sys.exit("The paperspec tools need %s %s; run them through pixi, or install %s with pip."
                  % ("the package" if len(_missing) == 1 else "the packages", " and ".join(_missing),
                     "it" if len(_missing) == 1 else "them"))

tools/oracle/crypto.py

@@ -1,22 +1,31 @@
 """Keyed hashing, key derivation, Merkle roots and Ed25519.
 
 Non-normative; see spec/05-hashes.md and spec/08-fidelity.md.
-HKDF and Ed25519 come from the `cryptography` package.
+HKDF and Ed25519 come from the `cryptography` package, imported on first use so that the hashing, Merkle
+and did:key functions work without it.
 """
 
 import hashlib
 
-from cryptography.exceptions import InvalidSignature
-from cryptography.hazmat.primitives import hashes
-from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey, Ed25519PublicKey
-from cryptography.hazmat.primitives.kdf.hkdf import HKDF
-
 from . import jcs
 
 
+def _cryptography():
+    """The `cryptography` modules HKDF and Ed25519 use: exceptions, hashes, ed25519 and hkdf."""
+    try:
+        from cryptography import exceptions
+        from cryptography.hazmat.primitives import hashes
+        from cryptography.hazmat.primitives.asymmetric import ed25519
+        from cryptography.hazmat.primitives.kdf import hkdf
+    except ImportError:
+        raise ImportError("HKDF and Ed25519 need the cryptography package: install paperspec[crypto]") from None
+    return exceptions, hashes, ed25519, hkdf
+
+
 def hkdf(ikm, salt, info, length=32):
     """HKDF-SHA-256, RFC 5869."""
-    return HKDF(algorithm=hashes.SHA256(), length=length, salt=salt, info=info).derive(ikm)
+    _, hashes, _, kdf = _cryptography()
+    return kdf.HKDF(algorithm=hashes.SHA256(), length=length, salt=salt, info=info).derive(ikm)
 
 
 def team_key(secret):
@@ -199,20 +208,23 @@ def verify_consistency(root_m, root_n, m, n, proof):
 
 def ed25519_public(secret):
     """The 32-byte public key of a 32-byte Ed25519 secret."""
-    return Ed25519PrivateKey.from_private_bytes(secret).public_key().public_bytes_raw()
+    _, _, ed25519, _ = _cryptography()
+    return ed25519.Ed25519PrivateKey.from_private_bytes(secret).public_key().public_bytes_raw()
 
 
 def ed25519_sign(secret, msg):
-    return Ed25519PrivateKey.from_private_bytes(secret).sign(msg)
+    _, _, ed25519, _ = _cryptography()
+    return ed25519.Ed25519PrivateKey.from_private_bytes(secret).sign(msg)
 
 
 def ed25519_verify(public, msg, signature):
     """True when `signature` is a valid Ed25519 signature of `msg` under `public`; never raises."""
     if len(public) != 32 or len(signature) != 64:
         return False
+    exceptions, _, ed25519, _ = _cryptography()
     try:
-        Ed25519PublicKey.from_public_bytes(public).verify(signature, msg)
+        ed25519.Ed25519PublicKey.from_public_bytes(public).verify(signature, msg)
-    except (InvalidSignature, ValueError):
+    except (exceptions.InvalidSignature, ValueError):
         return False
     return True
 

tools/oracle/schema.py

@@ -3,14 +3,12 @@
 Non-normative.
 The spec's schemas are restricted to the keywords below (spec/22-conformance.md) so that every 2020-12
 validator reads them alike; `lint` refuses any schema that leaves the subset.
+`jsonschema` and `referencing` are imported on first use, so `Store.load`, `deref` and `lint` work without them.
 """
 
 import json
 import os
 
-from jsonschema import Draft202012Validator
-from referencing import Registry, Resource
-
 KEYWORDS = {
     "$schema", "$id", "$ref", "$defs", "$comment", "title", "description", "examples", "default", "deprecated",
     "type", "enum", "const", "properties", "required", "additionalProperties", "patternProperties",
@@ -36,6 +34,8 @@ class Store:
 
     def registry(self):
         if self._registry is None:
+            from referencing import Registry, Resource
+
             names = sorted(f for f in os.listdir(self.root) if f.endswith(".schema.json"))
             self._registry = Registry().with_resources(
                 [(self.load(f)["$id"], Resource.from_contents(self.load(f))) for f in names])
@@ -43,6 +43,8 @@ class Store:
 
     def validator(self, name):
         if name not in self._validators:
+            from jsonschema import Draft202012Validator
+
             self._validators[name] = Draft202012Validator(self.load(name), registry=self.registry())
         return self._validators[name]