infra/paperspec
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat!: one paperspec command with subcommands, help and examples

Browse Source 
The six console scripts (paperspec-check, -build, -validate, -load,
-stress, -site) become subcommands of a single `paperspec` command:
paperspec validate, load, check, build, stress and site. `python -m
paperspec COMMAND` runs the same thing.

- paperspec/cli/main.py builds the whole parser, apart from the commands,
  so `paperspec --help` and `paperspec COMMAND --help` work without a
  checkout or the tools extra; a command's module is imported only when
  it runs.
- `paperspec` with no command prints the overview: what each command does,
  examples, and which work anywhere (validate, load) and which need a
  checkout (check, build, stress, site). Each command's help has a
  description with advice, and worked examples; a misspelled command gets
  a suggestion (argparse suggest_on_error); `--version` prints the version
  and the pinned Unicode version.
- Each command module exposes run(args) and returns the exit status.
  Arguments that parse but do not fit together raise UsageError, which
  prints the command's usage and exits 2: load now also refuses
  --built-from or --store without --bundle.
- The failure messages say what to do: install paperspec[tools], clone the
  repository, rerun stress with the printed seed, or run paperspec build
  when the rebuild check differs. check and stress say how many failures
  they did not list.
- check runs the rebuild as `python -m paperspec build`.
- pyproject installs one script, `paperspec`; the pixi tasks and the docs
  use the subcommands.

pixi run check passes (run here as `python -m paperspec check` with src/ on
the path); the rebuild is byte-identical. pixi.lock still needs
`pixi lock`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-15 09:28:30 -04:00
1 parent 3a5f28d010
commit 88379d152c
30 files changed
+385 -174
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

tools/README.md
+17 -16
View File
@@ -7,7 +7,8 @@ Implementers get the commands and the traps.
These commands are the repository's reference implementation.
They apply every rule of the specification, so the rest of the repository can be checked against them.
Two build and check the repository, one validates packages and proposed corrections, and one loads packages into SQLite, a database stored in a single file.
They are console scripts of the Python package `paperspec`, whose code is in [src/paperspec/](../src/paperspec/), with the commands in `src/paperspec/cli/`.
They are subcommands of one command, `paperspec`, which the Python package `paperspec` installs; its code is in [src/paperspec/](../src/paperspec/), with the commands in `src/paperspec/cli/`.
`paperspec --help` lists them and says where each works, and `paperspec COMMAND --help` gives a command's options, advice and examples.
They need Python 3.14 and two packages, `jsonschema` and `cryptography`, and are licensed Apache-2.0.
A package is the folder of files that holds one document, such as `fixtures/packages/hale2026harmonic-damb`.
@@ -16,14 +17,14 @@ A fixture is an example file or package under `fixtures/`.
| Command | Does |
| --- | --- |
| `paperspec-check` | Checks everything: schemas, records, packages and the zipped package, the broken packages, every vector, a byte-for-byte rebuild, the SQL (a database query language) round trip, the counts the prose states, and the prose. `--quick` skips the rebuild. |
| `paperspec-build` | Builds `fixtures/` and `vectors/` from the hand-written sources in `fixtures/src/` and the cases it defines. `--out DIR` builds into another folder. |
| `paperspec-validate` | Validates packages, as folders or as zipped packages (`.sct.zip`, unpacked under the rules of [spec/12 §12.2](../spec/12-package.md#122-paths)): `paperspec-validate PACKAGE [PACKAGE ...]`. With `--correction PROPOSALS` it runs proposed corrections through the gate against one package's current state, and prints the record each would append or the rule it breaks, without writing anything ([spec/11 §11.4](../spec/11-corrections.md#114-the-gate)). |
| `paperspec-load` | Validates packages, folders or zipped, then loads them into a SQLite database with the tables of `sql/`: `paperspec-load --sqlite lab.db PACKAGE [PACKAGE ...]`. With `--bundle --built-from REV` it builds a new bundle instead, with the full-text search index ([spec/13 §13.5](../spec/13-database.md#135-the-bundle)). |
| `paperspec-site` | Builds the website into `_site/`: every file of the repository at its own path, and each Markdown file rendered as a page beside it, with a vocabulary page, folder listings and a search index. It uses only the standard library. `pixi run check` builds it into a temporary folder and checks every link ([deploy/](../deploy/README.md)). |
| `paperspec-stress` | Stresses the oracle with random inputs from a seeded generator: `paperspec-stress [--seed N] [--rounds N]`. It checks the invariants the chapters state rather than the vectors' values, such as canonical JSON round trips, corrections keeping the five values consistent with a full recomputation, anchors always resolving to one of the five outcomes, the zip reader never writing outside its folder, and a mutated package always being refused. |
| `paperspec check` | Checks everything: schemas, records, packages and the zipped package, the broken packages, every vector, a byte-for-byte rebuild, the SQL (a database query language) round trip, the counts the prose states, and the prose. `--quick` skips the rebuild. |
| `paperspec build` | Builds `fixtures/` and `vectors/` from the hand-written sources in `fixtures/src/` and the cases it defines. `--out DIR` builds into another folder. |
| `paperspec validate` | Validates packages, as folders or as zipped packages (`.sct.zip`, unpacked under the rules of [spec/12 §12.2](../spec/12-package.md#122-paths)): `paperspec validate PACKAGE [PACKAGE ...]`. With `--correction PROPOSALS` it runs proposed corrections through the gate against one package's current state, and prints the record each would append or the rule it breaks, without writing anything ([spec/11 §11.4](../spec/11-corrections.md#114-the-gate)). |
| `paperspec load` | Validates packages, folders or zipped, then loads them into a SQLite database with the tables of `sql/`: `paperspec load --sqlite lab.db PACKAGE [PACKAGE ...]`. With `--bundle --built-from REV` it builds a new bundle instead, with the full-text search index ([spec/13 §13.5](../spec/13-database.md#135-the-bundle)). |
| `paperspec site` | Builds the website into `_site/`: every file of the repository at its own path, and each Markdown file rendered as a page beside it, with a vocabulary page, folder listings and a search index. It uses only the standard library. `pixi run check` builds it into a temporary folder and checks every link ([deploy/](../deploy/README.md)). |
| `paperspec stress` | Stresses the oracle with random inputs from a seeded generator: `paperspec stress [--seed N] [--rounds N]`. It checks the invariants the chapters state rather than the vectors' values, such as canonical JSON round trips, corrections keeping the five values consistent with a full recomputation, anchors always resolving to one of the five outcomes, the zip reader never writing outside its folder, and a mutated package always being refused. |
`paperspec-validate` and `paperspec-load` work anywhere the package is installed, with the schemas and SQL it carries.
`paperspec validate` and `paperspec load` work anywhere the package is installed, with the schemas and SQL it carries.
The other four work on a checkout of this repository: the one an editable install points at, or else the one around the current folder.
## When to run what
@@ -37,21 +38,21 @@ If you do not use pixi, install the checkout editable with its extras on Python
```sh
pip install -e ".[tools]"
paperspec-check # everything
paperspec-check --quick # everything except the rebuild
paperspec-build --out /tmp/rebuilt
paperspec-validate fixtures/packages/hale2026harmonic-damb
paperspec-load --sqlite lab.db fixtures/packages/hale2026harmonic-damb
paperspec check # everything
paperspec check --quick # everything except the rebuild
paperspec build --out /tmp/rebuilt
paperspec validate fixtures/packages/hale2026harmonic-damb
paperspec load --sqlite lab.db fixtures/packages/hale2026harmonic-damb
```
Each command is also a module, so `python -m paperspec.cli.check` runs the same program.
`python -m paperspec check` runs the same program as `paperspec check`, which helps where the installed command is not on the path.
A passing check ends with `All checks pass.`
The load command prints `hale2026harmonic-damb: 75 nodes`, one line per package, and writes `lab.db` with a folder `lab.db.assets/` beside it for files that have no table of their own.
It validates every package first and loads nothing if one fails, printing each failure as `package: where: rule: message`.
The full and checks-only forms of one document share its key, so they go into separate databases.
Each command prints its options with `--help` and refuses options it does not know.
`paperspec-build` rebuilds `fixtures/` and `vectors/` in place unless you pass `--out`.
`paperspec build` rebuilds `fixtures/` and `vectors/` in place unless you pass `--out`.
Run `pixi run check` before every commit.
Run `pixi run build` after you change a fixture source, a rule in `src/paperspec/` or a case in `src/paperspec/cli/build.py`, and commit the regenerated files with the change.
@@ -71,7 +72,7 @@ Canonical JSON goes through Python's built-in JSON encoder, and falls back to an
Hashes are kept incrementally.
A node's five values depend only on its own subtree, so after a correction the oracle rehashes only the nodes that changed and their ancestors, and a container's projections are its children's cached projections joined.
`paperspec-check` compares the result after every correction with a slow computation that projects each node from its whole subtree.
`paperspec check` compares the result after every correction with a slow computation that projects each node from its whole subtree.
Other repositories of the project do not reimplement the specification: they install this repository as the Python package `paperspec` (`pip install .`, or a git pin) and import the oracle from it, so that every key, identifier, canonical JSON string, package check and database form comes from one piece of code.
The library surface they use is `keys`, `text`, `jcs`, `package`, `db`, `schema` and `rights`, with the schemas, SQL and vectors as package data: `paperspec.resources.path("sql")` finds a folder in an installed wheel and in an editable install alike.