feat: publish the package from CI, and clean up its references

Publishing
- .gitea/workflows/publish.yml publishes paperspec to the instance's PyPI
  registry (/api/packages/infra/pypi) when a v* tag is pushed: it checks
  the repository, builds the sdist and wheel, refuses to upload unless the
  built version equals the tag, runs twine check --strict, and uploads
  with the secrets PACKAGE_USER and PACKAGE_TOKEN (--skip-existing, so a
  rerun on the same tag is safe). Run by hand, it refuses anything but a
  tag.
- pixi dev environment: twine >= 6.1 (the first that reads Metadata 2.4),
  and `dist` (sdist and wheel) replaces the `wheel` task.
- deploy/README.md: how to set up the secrets, release a version, and
  install from the registry with pixi, uv or pip, tying paperspec to the
  registry where the installer allows it.
- CONTRIBUTING.md "Releasing": a tag versions the package; a format
  change takes the new format version as the top CHANGELOG heading, and a
  code-only release takes the next patch as a subsection, as 0.1.1 does.
- pyproject: fallback_version 0+unknown for source trees without git; the
  license expression is Apache-2.0 AND CC0-1.0, for the code and the data
  the wheel carries.

Cleanup (from an audit of every reference to the package)
- spec/00 lists src/ (the package) and tools/ (how to run it), and no
  longer says the reference implementation is not a library.
- Stale explanations fixed in pixi.toml, .gitignore, the Caddyfile, the
  vectors, context, sql and tools READMEs, docs/examples.md, docs/faq.md,
  spec/01, docs/rationale.md and docs/comparison.md: the commands, what
  they need (the tools extra), the fixture folder hale2026harmonic-damb,
  and three lines that held two sentences.
- tools/README.md counts all six commands, says what needs which extra
  (package.validate and corrections.gate need [schema]), and that the
  vectors ship but the fixtures they name do not.
- The CLI's install and clone hints point at the registry instructions and
  an https clone URL; schema.py raises an ImportError naming
  paperspec[schema], as crypto.py does for [crypto].
- check's Unicode check now guards the package's pinned constant.
- site skips build/, dist/ and *.egg-info.
- CHANGELOG 0.1.1 describes the final state.

pixi run check passes (as `python -m paperspec check`); the rebuild is
byte-identical. pixi.lock needs `pixi lock` for twine before pushing: the
publish workflow installs with --locked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.gitea/workflows/publish.yml

@@ -0,0 +1,66 @@
+# Publishes the paperspec Python package to this instance's PyPI registry when a version tag (v0.1.1) is
+# pushed; see deploy/README.md for the secrets it needs and how to install from the registry.
+# The package version comes from the tag through setuptools-scm, so the job refuses to upload unless the
+# built version equals the tag. Run it by hand only on a tag, for example after adding the secrets.
+name: publish
+on:
+  push:
+    tags: ["v*"]
+  workflow_dispatch:
+permissions:
+  contents: read
+concurrency:
+  group: publish-${{ github.ref }}
+  cancel-in-progress: false
+defaults:
+  run:
+    shell: bash
+jobs:
+  publish:
+    runs-on: linux-base
+    container:
+      image: git.scient.ing/infra/linux-base:26.10.7
+    timeout-minutes: 30
+    env:
+      REGISTRY: ${{ github.server_url }}/api/packages/${{ github.repository_owner }}/pypi
+    steps:
+      - name: Require a version tag
+        run: |
+          if [[ "$GITHUB_REF" != refs/tags/v* ]]; then
+            echo "Publish runs on a version tag such as v0.1.1; $GITHUB_REF is not one."
+            exit 1
+          fi
+      # The whole history, so setuptools-scm finds the tag and computes the release version.
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          persist-credentials: false
+      - name: Check the repository
+        run: pixi run --locked check
+      - name: Build the sdist and wheel
+        run: pixi run --locked -e dev dist
+      - name: Confirm the built version is the tag
+        run: |
+          tag="${GITHUB_REF_NAME#v}"
+          wheels=(dist/paperspec-*.whl)
+          built="$(basename "${wheels[0]}" | cut -d- -f2)"
+          echo "tag $GITHUB_REF_NAME, built $built"
+          if [ "${#wheels[@]}" -ne 1 ] || [ "$built" != "$tag" ]; then
+            echo "The built version is not the tag's; is the tag on the commit this job checked out?"
+            exit 1
+          fi
+      - name: Check the package metadata
+        run: pixi run --locked -e dev twine check --strict dist/*
+      # --skip-existing makes a rerun on the same tag succeed instead of failing on the files already there.
+      - name: Upload to the package registry
+        env:
+          TWINE_USERNAME: ${{ secrets.PACKAGE_USER }}
+          TWINE_PASSWORD: ${{ secrets.PACKAGE_TOKEN }}
+          TWINE_REPOSITORY_URL: ${{ env.REGISTRY }}
+        run: |
+          if [ -z "$TWINE_USERNAME" ] || [ -z "$TWINE_PASSWORD" ]; then
+            echo "The secrets PACKAGE_USER and PACKAGE_TOKEN are not set; deploy/README.md says how."
+            exit 1
+          fi
+          pixi run --locked -e dev twine upload --non-interactive --skip-existing dist/*
+          echo "Published to $REGISTRY: pip install --index-url $REGISTRY/simple/ paperspec"

.gitignore

@@ -6,7 +6,7 @@ __pycache__/
 # the package version, written by setuptools-scm when the package is built or installed
 /src/paperspec/_version.py
 
-# the website, built by tools/site.py
+# the website, built by `paperspec site`
 /_site/
 
 # pixi environments
@@ -761,7 +761,7 @@ FodyWeavers.xsd
 
 # End of https://www.toptal.com/developers/gitignore/api/osx,python,pycharm,windows,visualstudio,visualstudiocode,zsh,jetbrains,jupyternotebooks
 
-# The Visual Studio rules above ignore every packages/ and log/ folder; these hold fixtures that tools/check.py
+# The Visual Studio rules above ignore every packages/ and log/ folder; these hold fixtures that `paperspec check`
 # compares byte for byte, so keep them.
 !/fixtures/packages/*
 !/fixtures/records/log/

CHANGELOG.md

@@ -28,7 +28,10 @@ Packaging, commands and lazy imports only; the formats stay at 0.1.0, and no com
 - The package version comes from git tags through setuptools-scm.
 - The base package has no dependencies and requires Python 3.14, the only version with the Unicode 16.0 character data the specification pins, and refuses to import under any other Unicode version; the extras `paperspec[schema]` (`jsonschema`, `referencing`), `paperspec[crypto]` (`cryptography`) and `paperspec[tools]` (both) cover `schema` and the HKDF and Ed25519 functions of `crypto`, which now import their packages on first use.
 - `paperspec.resources.path` finds the schemas, SQL and vectors in an installed wheel and in an editable install alike.
-- The programs in `tools/` become subcommands of one command, `paperspec`, in `src/paperspec/cli/`: `paperspec validate`, `load`, `check`, `build`, `stress` and `site`, which the pixi tasks and the workflow run, and which `python -m paperspec` also runs.
-- `paperspec --help` says which commands work anywhere and which need a checkout, and each command's `--help` gives advice and examples; a misspelled command gets a suggestion.
-- `src/` is Apache-2.0, as `tools/` was.
-- The pixi workspace installs the package editable, and a `dev` environment builds the wheel with `pixi run -e dev wheel`.
+- The programs in `tools/` become subcommands of one command, `paperspec`, in `src/paperspec/cli/`: `paperspec validate`, `load`, `check`, `build`, `stress` and `site`, which the pixi tasks and the workflows run, and which `python -m paperspec` also runs.
+  The scripts `tools/*.py` are gone, so `python3 tools/check.py` and the like no longer work; `tools/` keeps the page that documents the commands.
+- `paperspec --help` says which commands work anywhere and which need a checkout, each command's `--help` gives advice and examples, a misspelled command gets a suggestion, and `paperspec --version` prints the package version and the Unicode version it pins.
+- `paperspec load` refuses `--built-from` and `--store` without `--bundle`, where it used to ignore them.
+- `src/` is Apache-2.0, as `tools/` was, and the package metadata states `Apache-2.0 AND CC0-1.0` for the code and the data it carries.
+- The pixi workspace installs the package editable, and a `dev` environment builds the sdist and the wheel with `pixi run -e dev dist`.
+- Pushing a version tag publishes the package to the Gitea package registry through `.gitea/workflows/publish.yml` ([deploy/](deploy/README.md#publish-the-python-package)); a release tag versions the package, and [CONTRIBUTING.md](CONTRIBUTING.md#releasing) says how that relates to the format version.

CONTRIBUTING.md

@@ -34,7 +34,10 @@ A run kind needs its row in [section 3.5](spec/03-nodes.md#35-inline-runs) and t
 
 ## Releasing
 
-A release is a tag on `main` whose version is the top heading of [CHANGELOG.md](CHANGELOG.md).
+A release is a tag `vX.Y.Z` on `main`, and its version is the version of the Python package `paperspec`, which setuptools-scm reads from the tag.
+A release that changes the formats takes the new format version, which becomes the top heading of [CHANGELOG.md](CHANGELOG.md) and the version that `pixi.toml` and the registries state.
+A release that changes only the code takes the next patch version and records it as a subsection under the current format version, as `### 0.1.1` sits under `## 0.1.0`.
+Pushing the tag publishes the package to the Gitea package registry ([deploy/](deploy/README.md#publish-the-python-package)).
 Before tagging:
 
 1. `pixi run check` passes on Linux and macOS, which the workflow in `.gitea/workflows/` runs on every push.

README.md

@@ -83,8 +83,9 @@ The run ends with `All checks pass.`
 `pixi task list` shows the other tasks: `quick`, `build`, `lint`, `load`, `validate` and `stress`.
 `pixi run site` builds the website into `_site/` ([deploy/](deploy/README.md)).
 
-The tools need Python 3.14 and two packages, `jsonschema` and `cryptography`, which pixi installs along with markdownlint for the Markdown.
+The commands need Python 3.14 and the package's `tools` extra (`jsonschema`, `referencing` and `cryptography`), which pixi installs along with markdownlint for the Markdown.
 Without pixi, install the checkout as the package with `pip install -e ".[tools]"` on Python 3.14, then run `paperspec check`.
+`paperspec --help` lists the commands, and each command's `--help` gives advice and examples ([tools/](tools/README.md)).
 
 To explore a worked package, load the example paper (a synthetic chemistry paper with 12 corrections) into SQLite and query it; [docs/examples.md](docs/examples.md) has queries to start from.
 

context/README.md

@@ -96,11 +96,12 @@ Three of the nine fixture annotations have only an `AnchorSelector` (the section
 - Reader: Follow [chapter 6](../spec/06-anchors.md) to resolve an anchor.
   The file `vectors/resolve.jsonl` resolves seventeen anchors against `fixtures/packages/hale2026harmonic-damb` and gives every outcome.
 - JSON-LD processor: Use a JSON-LD 1.1 processor; the context relies on `@json` and `@version` 1.1.
-- Checker: `schemas/annotation.schema.json` and `schemas/layer.schema.json` validate the files. `paperspec check` also checks that each annotation's ID derives from its content and that the receipt at `fixtures/records/layer/receipt.json` covers each `confirmed` annotation.
+- Checker: `schemas/annotation.schema.json` and `schemas/layer.schema.json` validate the files.
+  `paperspec check` also checks that each annotation's ID derives from its content and that the receipt at `fixtures/records/layer/receipt.json` covers each `confirmed` annotation.
 
 The traps:
 
-- No test vector pins the context, and nothing in `tools/` reads `scienting-1.jsonld`.
+- No test vector pins the context, and nothing here runs a JSON-LD processor on it; `paperspec check` and `paperspec site` read only its term IRIs.
   The schemas require only that `@context` be an array of at least two non-empty strings; they do not check the URLs.
   A file with a wrong context can pass the checker and still expand wrongly.
 - `@context` is part of an annotation's content hash (the hash covers everything except `status`, `receipt` and `modified`).

deploy/Caddyfile

@@ -1,4 +1,4 @@
-# Serves the paperspec website built by `python3 tools/site.py` (the `_site/` folder), copied to /srv/www/paperspec.
+# Serves the paperspec website built by `paperspec site` (`pixi run site`, the `_site/` folder), copied to /srv/www/paperspec.
 # The site mirrors the repository, so every schema is served at its $id and the JSON-LD context at the address
 # layers name; this file sets the media types those addresses need and lets any page fetch them.
 paperspec.scient.ing {

deploy/README.md

@@ -1,6 +1,7 @@
-# Deploying the website
+# Deploying the website and the Python package
 
 This folder holds what it takes to serve the paperspec website at <https://paperspec.scient.ing>.
+The last sections say how the Python package `paperspec` is published to the Gitea package registry, and how other repositories install it from there.
 The site is static: `paperspec site` builds it into `_site/`, and any web server can serve that folder.
 
 ## What the site serves
@@ -52,7 +53,69 @@ On a runner without that mount the workflow still builds and checks the site, an
 Only changed files are copied, so every other file keeps its `Last-Modified` and `ETag` and caches stay valid across releases.
 The environment in `.pixi/envs` is cached between runs under the hash of `pixi.lock`, so a fresh container installs it only when the lock file changes.
 
-## Distributing packages
+## Publish the Python package
+
+The Python package `paperspec` holds the reference implementation and the `paperspec` command.
+The workflow `.gitea/workflows/publish.yml` publishes it to the PyPI registry of the Gitea instance, `https://git.scient.ing/api/packages/infra/pypi`, when a version tag such as `v0.1.1` is pushed.
+It checks the repository, builds the sdist and the wheel with `pixi run -e dev dist`, confirms that the version setuptools-scm computed is the tag's, checks the metadata with twine, and uploads both files.
+The package version is the version of this code; the formats keep the version that `CHANGELOG.md` and `pixi.toml` state.
+
+### Set it up once
+
+1. Pick the account the workflow publishes as, a bot account or your own, and make it a member of a team in the `infra` organization with write access to packages.
+2. Signed in as that account, create an access token under Settings, Applications, with the `write:package` scope.
+3. In this repository's Settings, Actions, Secrets, add `PACKAGE_USER` with the account's user name and `PACKAGE_TOKEN` with the token.
+   Organization secrets work as well, for every repository in `infra`.
+   Gitea refuses secret names that start with `GITEA_` or `GITHUB_`.
+4. After the first upload, link the package to this repository in the package's settings, so it shows on the repository's Packages tab.
+
+### Release a version
+
+Tag a commit on `main` and push the tag:
+
+```sh
+git tag -a v0.1.1 -m "Release the paperspec package, version 0.1.1"
+git push origin v0.1.1
+```
+
+The job refuses to upload when the built version differs from the tag, for example when the tag is not on the commit it checked out.
+The registry never replaces a version, so to publish a fix, push a new tag.
+A rerun on the same tag skips the files already uploaded, which makes it safe to rerun a job that failed on its secrets: open the workflow under Actions and run it on the tag.
+
+### Install from the registry
+
+The registry does not mirror PyPI, so an installer needs two indexes: the registry for `paperspec`, and PyPI for `jsonschema`, `referencing` and `cryptography`.
+Tie `paperspec` to the registry wherever the installer allows it, so that a package of the same name on PyPI is never chosen instead.
+
+With pixi, in `pixi.toml`:
+
+```toml
+[pypi-dependencies]
+paperspec = { version = ">=0.1.1", extras = ["tools"], index = "https://git.scient.ing/api/packages/infra/pypi/simple" }
+```
+
+With uv, in `pyproject.toml`:
+
+```toml
+[[tool.uv.index]]
+name = "scienting"
+url = "https://git.scient.ing/api/packages/infra/pypi/simple"
+explicit = true
+
+[tool.uv.sources]
+paperspec = { index = "scienting" }
+```
+
+With pip, which cannot tie a package to an index and takes the highest version from either:
+
+```sh
+pip install --index-url https://git.scient.ing/api/packages/infra/pypi/simple/ --extra-index-url https://pypi.org/simple/ "paperspec[tools]"
+```
+
+If the organization or the package is private, the installer also needs a token with the `read:package` scope, for example in a `~/.netrc` entry for `git.scient.ing`.
+A git pin to a commit, as `pixi.toml` and `pyproject.toml` can also express, keeps working and needs no registry.
+
+## Distributing document packages
 
 A zipped package is an ordinary file, and the site serves the fixtures as such.
 For a team's own packages, the Gitea instance speaks the OCI distribution protocol, so packages can be pushed to and pulled from its package registry as OCI artifacts with ORAS, and signed there with cosign; the media type of the artifact is the package's, `application/vnd.scienting.doc+zip`.

docs/comparison.md

@@ -429,7 +429,7 @@ Keep using it as a reading and distribution format.
 The specification is version 0.1.0.
 `registry/formats.json` lists 32 formats: 19 are `candidate` (complete, with vectors and fixtures, ready to implement) and 13 are `provisional` (complete; expected to change as implementations gain experience): project, bundle, dataset, digitization, view, snapshot, draft-ids, exercise, submission, grade, checklist, log-entry and checkpoint ([spec/00 §0.4](../spec/00-introduction.md#04-status)).
 The repository holds 24 chapters, 42 JSON Schemas, 25 vector files, five valid fixture packages, one of them also zipped, and 31 invalid ones, each naming the rule it breaks, and a reference checker, `paperspec check`.
-Licenses: prose CC BY 4.0; schemas, SQL, the JSON-LD context, registries, vectors and fixtures CC0 1.0; tools Apache-2.0 ([README.md](../README.md#licenses)).
+Licenses: prose CC BY 4.0; schemas, SQL, the JSON-LD context, registries, vectors and fixtures CC0 1.0; the code in `src/` Apache-2.0 ([README.md](../README.md#licenses)).
 
 *No adoption claim.*
 The repository names no outside adopter and no independent implementation, and this page claims none.

docs/examples.md

@@ -9,7 +9,7 @@ The formats are version 0.1.0, and each is `candidate` or `provisional` in `regi
 
 Run every command from the root of the repository, <https://git.scient.ing/infra/paperspec>.
 With [pixi](https://pixi.sh), `pixi run <task>` sets up the environment first.
-Every command also works without pixi, with Python 3.14 and two packages, `jsonschema` and `cryptography` installed and standard command-line tools (the `sed` command in the last section is the GNU version of `sed`).
+Every command also works without pixi once the checkout is installed with `pip install -e ".[tools]"` on Python 3.14, given standard command-line tools (the `sed` command in the last section is the GNU version of `sed`).
 
 ## Run the checks
 

docs/faq.md

@@ -124,8 +124,9 @@ Propose either with the vector or fixture that shows it.
 A new kind is a minor version ([spec/00 §0.3](../spec/00-introduction.md#03-format-identifiers-and-versions)).
 See [CONTRIBUTING.md](../CONTRIBUTING.md).
 
-## What are the files in `tools/` for?
+## What are the `paperspec` commands for?
 
-`paperspec build` builds the fixtures and vectors. `paperspec check` recomputes and checks every one, and `paperspec load` loads packages into SQLite.
+`paperspec build` builds the fixtures and vectors.
+`paperspec check` recomputes and checks every one, and `paperspec load` loads packages into SQLite.
 The oracle's modules in `src/paperspec/` compute every expected value, so that none is typed by hand.
 The other repositories of the project install them as the Python package `paperspec`; independent implementations follow the specification ([tools/README.md](../tools/README.md)).

docs/rationale.md

@@ -112,7 +112,7 @@ Only the three text values, `text_hash`, `match_hash` and `align_key`, normalize
 ### The character data is pinned to one Unicode version
 
 Normalization and General Categories change between Unicode versions, and both enter every hash.
-The chapters pin Unicode 16.0, the reference tools refuse to run on another version, and moving to a later one is a change to every text-hashing format ([spec/01 §1.4](../spec/01-conventions.md#14-text-normalization-for-hashing)).
+The chapters pin Unicode 16.0, the reference implementation refuses to import on another version, and moving to a later one is a change to every text-hashing format ([spec/01 §1.4](../spec/01-conventions.md#14-text-normalization-for-hashing)).
 
 ### Hashes that leave the team are keyed
 

pixi.toml

@@ -20,19 +20,21 @@ markdownlint-cli2 = ">=0.23.3"
 nodejs = ">=22"
 cryptography = ">=46"
 
-# The reference implementation in src/paperspec, installed editable so that `import paperspec` works in
-# every pixi environment; the tools put src/ first on the path themselves, so they also run without pixi.
+# The reference implementation in src/paperspec, installed editable so that `import paperspec` and the
+# `paperspec` command work in every pixi environment; without pixi, `pip install -e ".[tools]"` does the same.
 [pypi-dependencies]
 paperspec = { path = ".", editable = true }
 
-# What building the package needs: pixi run -e dev wheel
+# What building and publishing the package needs: pixi run -e dev dist; see deploy/README.md.
+# twine 6.1 is the first that reads the license metadata (Metadata-Version 2.4) setuptools writes.
 [feature.dev.dependencies]
 setuptools = ">=80"
 setuptools-scm = ">=8"
 python-build = ">=1.2"
+twine = ">=6.1"
 
 [feature.dev.tasks]
-wheel = { cmd = "python -m build --wheel --no-isolation --outdir dist", description = "Build the paperspec wheel into dist/" }
+dist = { cmd = "python -m build --sdist --wheel --no-isolation --outdir dist", description = "Build the paperspec sdist and wheel into dist/" }
 
 [environments]
 default = { solve-group = "default" }

pyproject.toml

@@ -7,7 +7,8 @@ name = "paperspec"
 dynamic = ["version"]
 description = "The scienting formats: the reference implementation of every rule, with the JSON Schemas, SQL and test vectors as package data"
 readme = "README.md"
-license = "Apache-2.0"
+# The code (src/) is Apache-2.0 and the schemas, SQL and vectors it carries are CC0-1.0, as LICENSE.md says.
+license = "Apache-2.0 AND CC0-1.0"
 license-files = ["LICENSE.md"]
 authors = [{ name = "Scientific Computing Studio", email = "maintainer@scient.ing" }]
 # Python 3.14 is the only version with the Unicode 16.0 character data the specification pins (spec/01 §1.4);
@@ -57,9 +58,11 @@ packages = ["paperspec", "paperspec.cli", "paperspec.schemas", "paperspec.sql",
 "paperspec.sql" = ["*.sql"]
 "paperspec.vectors" = ["*.jsonl"]
 
-# The package version comes from git tags (v0.1.0 builds as 0.1.0, later commits as 0.1.1.devN). It is the
-# version of this code, not of the formats, which CHANGELOG.md and pixi.toml carry.
+# The package version comes from git tags (v0.1.1 builds as 0.1.1, later commits as 0.1.2.devN); CONTRIBUTING.md
+# says how it relates to the format version that CHANGELOG.md and pixi.toml carry. A source tree without git
+# history, such as a downloaded archive, builds as 0+unknown instead of failing.
 [tool.setuptools_scm]
+fallback_version = "0+unknown"
 version_file = "src/paperspec/_version.py"
 version_scheme = "guess-next-dev"
 local_scheme = "node-and-timestamp"

spec/00-introduction.md

@@ -82,8 +82,9 @@ The other folders help people use the specification.
 | `registry/` | Formats, rule IDs, stamp rules and the default stage requirements. | Yes |
 | `vectors/` | Test vectors, one case per line. | Yes |
 | `fixtures/` | Valid packages and records, invalid packages, and the keys that signed them. | Yes |
-| `tools/` | The programs that compute and check the vectors and fixtures, and load packages into SQLite. | No |
+| `src/` | The Python package `paperspec`: the reference implementation, and the `paperspec` command that computes and checks the vectors and fixtures and loads packages into SQLite. | No |
+| `tools/` | How to run those commands. | No |
 | `docs/` | Guides for people getting to know the formats. | No |
 
-The programs in `tools/` compute every expected value, so that none is typed by hand.
-They are not a library, and implementations conform to the chapters and the vectors.
+The reference implementation in `src/paperspec/` computes every expected value, so that none is typed by hand.
+Other repositories of the project import it, but implementations conform to the chapters and the vectors, not to it.

spec/01-conventions.md

@@ -46,7 +46,7 @@ Stored text is kept exactly as witnessed; normalization applies only when a valu
 The vectors in `normalize.jsonl` cover the cases, including the angstrom sign, which NFC turns into Å.
 
 Normalization forms, General Categories and the `White_Space` property are those of Unicode 16.0.
-A later Unicode version can change a decomposition or a category, and with it a hash, so an implementation states the Unicode version its character data comes from, the reference tools refuse to run on another, and moving the formats to a later version is a change to every format that hashes text ([0.3](00-introduction.md#03-format-identifiers-and-versions)).
+A later Unicode version can change a decomposition or a category, and with it a hash, so an implementation states the Unicode version its character data comes from, the reference implementation refuses to import on another, and moving the formats to a later version is a change to every format that hashes text ([0.3](00-introduction.md#03-format-identifiers-and-versions)).
 
 ## 1.5 Character offsets
 

sql/README.md

@@ -57,7 +57,8 @@ On `fixtures/packages/hale2026harmonic-damb` it returns one row, `hca6dc5`: the
 - Names: no table, column or index name is reserved in SQLite, PostgreSQL or MySQL (Dolt is MySQL-compatible).
   The formats' `key`, `index`, `at`, `by`, `before` and `after` become `doc_key`, `idx`, `made_at`, `signer`, `before_hash` and `after_hash`.
 - Round trip: each row keeps every property of its record, so unloading restores the same bytes and the same manifest hash.
-  Unloading copies the files with no table from the asset store, which must travel with the database. `paperspec check` tests this for every fixture package.
+  Unloading copies the files with no table from the asset store, which must travel with the database.
+  `paperspec check` tests this for every fixture package.
 - Derived tables: `citations`, `receipt_nodes` and `annotation_targets` are built on load for queries and are not part of the package.
 
 ## Status

src/paperspec/__init__.py

@@ -1,10 +1,10 @@
 """The reference implementation of the scienting formats: the computations behind every expected value
-in this repository; see tools/README.md.
+in the paperspec repository; see https://paperspec.scient.ing/tools/.
 
-The modules other programs use (`keys`, `text`, `jcs`, `package`, `db`, `rights`, ...) need only the
-standard library.
-`schema` needs `paperspec[schema]` and the HKDF and Ed25519 functions of `crypto` need `paperspec[crypto]`;
-both import their packages on first use.
+Every module imports with the standard library alone. Schema validation (`schema`, and with it
+`package.validate` and `corrections.gate`) needs `paperspec[schema]`, and HKDF and Ed25519 in `crypto`, which
+the signature and receipt checks use, need `paperspec[crypto]`; both import their packages on first use, and
+`paperspec[tools]` installs both.
 The schemas, SQL and vectors travel with the package: `paperspec.resources.path("sql")`.
 The commands are in `paperspec.cli`.
 

src/paperspec/cli/__init__.py

@@ -10,7 +10,8 @@ import importlib.util
 import os
 import sys
 
-REPOSITORY = "ssh://git@git.scient.ing/infra/paperspec.git"
+REPOSITORY = "https://git.scient.ing/infra/paperspec.git"
+INSTALL = "https://paperspec.scient.ing/deploy/#install-from-the-registry"
 
 
 class UsageError(Exception):
@@ -21,9 +22,10 @@ def require(*modules):
     """Exit with what to install when a package a command needs is missing."""
     missing = [m for m in modules if importlib.util.find_spec(m) is None]
     if missing:
-        sys.exit("This command needs %s %s.\nInstall them with: pip install \"paperspec[tools]\"\n"
-                 "In a checkout of the repository, pixi installs them: pixi run COMMAND"
-                 % ("the package" if len(missing) == 1 else "the packages", " and ".join(missing)))
+        sys.exit("This command needs %s %s, which the package's tools extra installs.\n"
+                 "Install paperspec[tools] from the package registry: %s\n"
+                 "In a checkout of the repository, pip install -e \".[tools]\" or pixi run COMMAND installs them."
+                 % ("the package" if len(missing) == 1 else "the packages", " and ".join(missing), INSTALL))
 
 
 def _is_checkout(d):

src/paperspec/cli/check.py

@@ -119,9 +119,12 @@ def check_registry():
                     ok("registry", rule in rules, "src/paperspec/%s raises %s, which is not in rules.json" % (f, rule))
     for v in ("text-hash", "match-hash", "align-key", "own-hash", "node-hash"):
         ok("registry", "doc.hash." + v in rules, "doc.hash.%s is not in rules.json" % v)
-    # The character data the hashes depend on is pinned to one Unicode version (spec/01 §1.4).
-    ok("registry", unicodedata.unidata_version == "16.0.0",
-       "the tools run on Unicode %s, but the specification pins 16.0" % unicodedata.unidata_version)
+    # The character data the hashes depend on is pinned to one Unicode version (spec/01 §1.4); the package
+    # refuses to import on any other, so this guards the pin itself against changing without the chapters.
+    import paperspec
+    ok("registry", paperspec.UNICODE_VERSION == "16.0.0" == unicodedata.unidata_version,
+       "the package pins Unicode %s and runs on %s, but the specification pins 16.0"
+       % (paperspec.UNICODE_VERSION, unicodedata.unidata_version))
     for d in sorted(os.listdir(p("fixtures", "invalid"))):
         ok("registry", js(p("fixtures", "invalid", d, "expect.json"))["rule"] in rules, "%s: rule is not in rules.json" % d)
 

src/paperspec/cli/main.py

@@ -27,13 +27,13 @@ examples:
   paperspec check --quick                           in a checkout: check everything but the rebuild
 
 where each command works:
-  validate and load work wherever paperspec is installed with its tools extra:
-      pip install "paperspec[tools]"
+  validate and load work wherever paperspec is installed with its tools extra, paperspec[tools];
+  installing from the package registry: %s/deploy/#install-from-the-registry
   check, build, stress and site work on a checkout of the paperspec repository,
   where pixi also runs them as tasks: pixi run check
 
 Run "paperspec COMMAND --help" for a command's options, advice and examples.
-"""
+""" % SITE
 
 PACKAGE_ADVICE = """\
 A package is a folder, or a zipped package (.sct.zip) that is unpacked under the rules of

src/paperspec/cli/site.py

@@ -19,7 +19,7 @@ import shutil
 from paperspec.cli import UsageError, checkout
 
 ROOT = checkout()
-SKIP_DIRS = {".git", ".pixi", ".claude", "_site", "__pycache__", "node_modules", ".gitea", ".vscode", ".idea"}
+SKIP_DIRS = {".git", ".pixi", ".claude", "_site", "__pycache__", "node_modules", ".gitea", ".vscode", ".idea", "build", "dist"}
 SKIP_FILES = {".DS_Store"}
 NAME = "paperspec"
 REPO = "https://git.scient.ing/infra/paperspec"
@@ -571,7 +571,8 @@ def build(out, base_url=BASE_URL):
     os.makedirs(out)
     mds = []
     for d, dirs, fs in os.walk(ROOT):
-        dirs[:] = sorted(x for x in dirs if x not in SKIP_DIRS and not (x.startswith(".") and x != ".well-known"))
+        dirs[:] = sorted(x for x in dirs if x not in SKIP_DIRS and not x.endswith(".egg-info")
+                         and not (x.startswith(".") and x != ".well-known"))
         reld = os.path.relpath(d, ROOT)
         reld = "" if reld == "." else reld.replace(os.sep, "/")
         if os.path.abspath(d) == os.path.abspath(out):

src/paperspec/schema.py

@@ -23,6 +23,17 @@ class SchemaError(Exception):
     pass
 
 
+def _validation():
+    """The validator and registry classes from `jsonschema` and `referencing`, imported on first use."""
+    try:
+        from jsonschema import Draft202012Validator
+        from referencing import Registry, Resource
+    except ImportError:
+        raise ImportError("JSON Schema validation needs the jsonschema and referencing packages: "
+                          "install paperspec[schema]") from None
+    return Draft202012Validator, Registry, Resource
+
+
 class Store:
     """The schemas of one folder, linted on load, with a compiled validator per schema."""
 
@@ -34,8 +45,7 @@ class Store:
 
     def registry(self):
         if self._registry is None:
-            from referencing import Registry, Resource
-
+            _, Registry, Resource = _validation()
             names = sorted(f for f in os.listdir(self.root) if f.endswith(".schema.json"))
             self._registry = Registry().with_resources(
                 [(self.load(f)["$id"], Resource.from_contents(self.load(f))) for f in names])
@@ -43,8 +53,7 @@ class Store:
 
     def validator(self, name):
         if name not in self._validators:
-            from jsonschema import Draft202012Validator
-
+            Draft202012Validator, _, _ = _validation()
             self._validators[name] = Draft202012Validator(self.load(name), registry=self.registry())
         return self._validators[name]
 

tools/README.md

@@ -6,10 +6,10 @@ Implementers get the commands and the traps.
 
 These commands are the repository's reference implementation.
 They apply every rule of the specification, so the rest of the repository can be checked against them.
-Two build and check the repository, one validates packages and proposed corrections, and one loads packages into SQLite, a database stored in a single file.
+Two build and check the repository, one validates packages and proposed corrections, one loads packages into SQLite, a database stored in a single file, one stresses the reference implementation with random inputs, and one builds the website.
 They are subcommands of one command, `paperspec`, which the Python package `paperspec` installs; its code is in [src/paperspec/](../src/paperspec/), with the commands in `src/paperspec/cli/`.
 `paperspec --help` lists them and says where each works, and `paperspec COMMAND --help` gives a command's options, advice and examples.
-They need Python 3.14 and two packages, `jsonschema` and `cryptography`, and are licensed Apache-2.0.
+They need Python 3.14 and the package's `tools` extra (`jsonschema`, `referencing` and `cryptography`), except `paperspec site`, which needs only the standard library, and they are licensed Apache-2.0.
 
 A package is the folder of files that holds one document, such as `fixtures/packages/hale2026harmonic-damb`.
 A vector is a test case: an input and the exact output every implementation must compute (see [the vectors](../vectors/README.md)).
@@ -24,7 +24,7 @@ A fixture is an example file or package under `fixtures/`.
 | `paperspec site` | Builds the website into `_site/`: every file of the repository at its own path, and each Markdown file rendered as a page beside it, with a vocabulary page, folder listings and a search index. It uses only the standard library. `pixi run check` builds it into a temporary folder and checks every link ([deploy/](../deploy/README.md)). |
 | `paperspec stress` | Stresses the oracle with random inputs from a seeded generator: `paperspec stress [--seed N] [--rounds N]`. It checks the invariants the chapters state rather than the vectors' values, such as canonical JSON round trips, corrections keeping the five values consistent with a full recomputation, anchors always resolving to one of the five outcomes, the zip reader never writing outside its folder, and a mutated package always being refused. |
 
-`paperspec validate` and `paperspec load` work anywhere the package is installed, with the schemas and SQL it carries.
+`paperspec validate` and `paperspec load` work anywhere the package is installed with its `tools` extra, using the schemas and SQL it carries.
 The other four work on a checkout of this repository: the one an editable install points at, or else the one around the current folder.
 
 ## When to run what
@@ -49,7 +49,7 @@ paperspec load --sqlite lab.db fixtures/packages/hale2026harmonic-damb
 
 A passing check ends with `All checks pass.`
 The load command prints `hale2026harmonic-damb: 75 nodes`, one line per package, and writes `lab.db` with a folder `lab.db.assets/` beside it for files that have no table of their own.
-It validates every package first and loads nothing if one fails, printing each failure as `package: where: rule: message`.
+It validates every package first and loads nothing if one fails, printing each failure as `package: where: rule: message`, or `package: rule: message` for a zipped package that cannot be unpacked.
 The full and checks-only forms of one document share its key, so they go into separate databases.
 Each command prints its options with `--help` and refuses options it does not know.
 `paperspec build` rebuilds `fixtures/` and `vectors/` in place unless you pass `--out`.
@@ -74,10 +74,13 @@ Hashes are kept incrementally.
 A node's five values depend only on its own subtree, so after a correction the oracle rehashes only the nodes that changed and their ancestors, and a container's projections are its children's cached projections joined.
 `paperspec check` compares the result after every correction with a slow computation that projects each node from its whole subtree.
 
-Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec` (`pip install .`, or a git pin) and import the oracle from it, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
+Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec`, from the Gitea package registry or as a git pin ([deploy/](../deploy/README.md#install-from-the-registry)), and import the oracle from it, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
 The library surface they use is `keys`, `text`, `jcs`, `package`, `db`, `schema` and `rights`, with the schemas, SQL and vectors as package data: `paperspec.resources.path("sql")` finds a folder in an installed wheel and in an editable install alike.
+The vectors travel with the package, but `resolve.jsonl`, `grading.jsonl` and `targets.jsonl` name a fixture package, and the fixtures stay in the repository.
 The package needs Python 3.14, the only version with the Unicode 16.0 character data the specification pins, and refuses to import on any other.
-Otherwise it needs only the standard library, except that `schema` needs `paperspec[schema]` (`jsonschema` and `referencing`) and the HKDF and Ed25519 functions of `crypto` need `paperspec[crypto]`; `paperspec[tools]` installs both.
-The package version comes from git tags through setuptools-scm, and `pixi run -e dev wheel` builds the wheel into `dist/`.
+Importing any of its modules needs only the standard library.
+Schema validation (`schema`, and with it `package.validate` and `corrections.gate`) needs `paperspec[schema]` (`jsonschema` and `referencing`), and HKDF and Ed25519 in `crypto`, which the signature and receipt checks use, need `paperspec[crypto]`.
+`paperspec[tools]` installs both, and is what a program that validates packages needs.
+The package version comes from git tags through setuptools-scm, and `pixi run -e dev dist` builds the sdist and the wheel into `dist/`.
 When `scienting-core` exists it takes this role behind the same vectors, and the import changes without any computed value changing.
 Independent implementations follow the [specification](../spec/README.md) and test themselves against the [vectors](../vectors/README.md).

vectors/README.md

@@ -73,18 +73,18 @@ Some files change the procedure:
   Compute the field for each listed case and compare across the group.
 - `fingerprints.jsonl` needs the download-stamp rules in [registry/stamps/scienting-stamps-1.json](../registry/stamps/scienting-stamps-1.json), which each case names in `rules`.
 - `resolve.jsonl`, `grading.jsonl` and `targets.jsonl` name a package by a path from the repository root.
-  All three use `fixtures/packages/hale2026harmonic`.
+  All three use `fixtures/packages/hale2026harmonic-damb`.
 
 Several files also mix more than one kind of case, told apart by their fields.
 `ids.jsonl` has minting cases and letter-first cases, `keyed.jsonl` and `signatures.jsonl` have a case for each derivation or record, `spotcheck.jsonl` has sample-size cases and one full sample, and `rights.jsonl` has stage checks (a `stage` field), backend checks (a `backend` field) and one combining case (neither).
 
-The reference checker in [tools/](../tools/README.md) runs all twenty-five files, together with the rest of the repository's checks:
+The reference checker, `paperspec check` ([tools/](../tools/README.md)), runs all twenty-five files, together with the rest of the repository's checks:
 
 ```sh
 paperspec check --quick
 ```
 
-It needs Python 3.14 and two packages, `jsonschema` and `cryptography`, and has no option to run one file.
+It runs in a checkout, through pixi (`pixi run quick`) or after `pip install -e ".[tools]"` on Python 3.14, and has no option to run one file.
 It tests the reference implementation in `src/paperspec/` against the vectors.
 It does not test yours.