feat!: pin Python 3.14 and Unicode 16.0, and make the tools commands

Python and Unicode
- requires-python is >=3.14,<3.15: 3.14 is the only Python with the
  Unicode 16.0 data the specification pins (3.13 has 15.1, 3.15 has 17.0).
- `import paperspec` raises ImportError unless unicodedata reports
  paperspec.UNICODE_VERSION (16.0.0), so a source checkout or an install
  that bypasses requires-python cannot compute values on other data.

Commands
- tools/{check,build,validate,load,stress,site}.py move with history to
  src/paperspec/cli/ and install as paperspec-check, -build, -validate,
  -load, -stress and -site; each also runs as `python -m paperspec.cli.X`.
- paperspec.cli.require() exits with "install paperspec[tools]" when
  cryptography or jsonschema is missing; paperspec.cli.checkout() finds
  the repository the check, build, stress and site commands work on (the
  editable install's own checkout, else the one around the current
  folder) and exits with a message outside one. tools/__init__.py is gone.
- validate and load read the schemas and SQL through the new
  paperspec.resources.path(), which finds them in a wheel and in an
  editable install alike, so they work anywhere the package is installed.
- check runs the rebuild as `python -m paperspec.cli.build`.
- build still writes the provenance tool label "paperspec tools/build.py":
  it is part of every hashed and signed fixture.
- pixi tasks and the check workflow run the commands; tools/README.md
  documents them, and the other docs name the commands instead of paths.

License
- LICENSE.md gives `src/` Apache-2.0 alongside `tools/`. Since the
  previous commit moved the code to src/, no row had covered it.

pixi run check passes (run as `python -m paperspec.cli.check` with src/ on
the path, since this environment cannot install the entry points); the
byte-for-byte rebuild matches, so no fixture or vector changed.
pixi.lock still needs `pixi lock`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-15 09:17:27 -04:00
1 parent cfe1f39099
commit 607062056f
33 files changed
+227 -149

No files matched your search

+1 -1
View File
@@ -428,7 +428,7 @@ Keep using it as a reading and distribution format.
*Status.*
The specification is version 0.1.0.
`registry/formats.json` lists 32 formats: 19 are `candidate` (complete, with vectors and fixtures, ready to implement) and 13 are `provisional` (complete; expected to change as implementations gain experience): project, bundle, dataset, digitization, view, snapshot, draft-ids, exercise, submission, grade, checklist, log-entry and checkpoint ([spec/00 §0.4](../spec/00-introduction.md#04-status)).
The repository holds 24 chapters, 42 JSON Schemas, 25 vector files, five valid fixture packages, one of them also zipped, and 31 invalid ones, each naming the rule it breaks, and a reference checker, `tools/check.py`.
The repository holds 24 chapters, 42 JSON Schemas, 25 vector files, five valid fixture packages, one of them also zipped, and 31 invalid ones, each naming the rule it breaks, and a reference checker, `paperspec-check`.
Licenses: prose CC BY 4.0; schemas, SQL, the JSON-LD context, registries, vectors and fixtures CC0 1.0; tools Apache-2.0 ([README.md](../README.md#licenses)).
*No adoption claim.*
+5 -5
View File
@@ -19,10 +19,10 @@ This shows how to confirm that the repository's specification, fixtures and test
pixi run check
```
`pixi run check` lints the Markdown, then runs `python tools/check.py`, the repository's reference checker.
`pixi run check` lints the Markdown, then runs `paperspec-check`, the repository's reference checker.
Each section prints how many checks it ran, and the last line is `All checks pass.`
`pixi run quick`, or `python3 tools/check.py --quick`, skips the slowest part, which rebuilds everything and compares it byte for byte.
Without pixi, the Markdown lint is a separate step (`markdownlint-cli2`); `tools/check.py` does not run it.
`pixi run quick`, or `paperspec-check --quick`, skips the slowest part, which rebuilds everything and compares it byte for byte.
Without pixi, the Markdown lint is a separate step (`markdownlint-cli2`); `paperspec-check` does not run it.
## Recompute a hash by hand
@@ -146,7 +146,7 @@ This shows that the same packages can live in a database and be queried with SQL
pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic-damb fixtures/packages/okafor2025solvent-acth fixtures/packages/writer2027fixed-ynr7.v3
```
Without pixi, run `python3 tools/load.py` with the same arguments.
Without pixi, run `paperspec-load` with the same arguments.
```text
hale2026harmonic-damb: 75 nodes
@@ -154,7 +154,7 @@ okafor2025solvent-acth: 15 nodes
writer2027fixed-ynr7.v3: 15 nodes
```
The tables are those of [sql/doc.sql](../sql/doc.sql) and [sql/store.sql](../sql/store.sql), which run unchanged on Dolt and PostgreSQL as well ([chapter 13](../spec/13-database.md)); `tools/check.py` exercises them on SQLite only, and this repository has not yet run them on Dolt or PostgreSQL.
The tables are those of [sql/doc.sql](../sql/doc.sql) and [sql/store.sql](../sql/store.sql), which run unchanged on Dolt and PostgreSQL as well ([chapter 13](../spec/13-database.md)); `paperspec-check` exercises them on SQLite only, and this repository has not yet run them on Dolt or PostgreSQL.
Here are four queries, in Python's `sqlite3` so that nothing else is needed.
```python
+2 -2
View File
@@ -107,7 +107,7 @@ A `candidate` format is complete, with vectors and fixtures, and ready to implem
A `provisional` one is complete and expected to change as implementations gain experience.
The provisional formats are dataset, digitization, view, project, bundle, snapshot, draft-ids, exercise, submission, grade, checklist, log-entry and checkpoint.
The repository holds a reference checker (`tools/check.py`), 25 vector files, five valid packages, one of them also as a zip, and 31 deliberately broken packages ([spec/00 §0.4](../spec/00-introduction.md#04-status), [spec/22](../spec/22-conformance.md)).
The repository holds a reference checker (`paperspec-check`), 25 vector files, five valid packages, one of them also as a zip, and 31 deliberately broken packages ([spec/00 §0.4](../spec/00-introduction.md#04-status), [spec/22](../spec/22-conformance.md)).
## What does it not do?
@@ -126,6 +126,6 @@ See [CONTRIBUTING.md](../CONTRIBUTING.md).
## What are the files in `tools/` for?
`build.py` builds the fixtures and vectors. `check.py` recomputes and checks every one, and `load.py` loads packages into SQLite.
`paperspec-build` builds the fixtures and vectors. `paperspec-check` recomputes and checks every one, and `paperspec-load` loads packages into SQLite.
The oracle's modules in `src/paperspec/` compute every expected value, so that none is typed by hand.
The other repositories of the project install them as the Python package `paperspec`; independent implementations follow the specification ([tools/README.md](../tools/README.md)).
+3 -3
View File
@@ -54,7 +54,7 @@ It appears inside hashed data and test vectors, and it is separate from the repo
| [spec/](../spec/README.md) | The specification, in 24 chapters (00 to 23). Normative. |
| [schemas/](../schemas/README.md) | 42 JSON (plain-text data format) Schemas for the files and records. |
| [registry/](../registry/README.md) | The list of formats, the rule IDs a checker reports, download-stamp rules, and default rights requirements. |
| [tools/](../tools/README.md) | Reference programs: `check.py` checks everything, `build.py` rebuilds the fixtures and vectors, and `load.py` loads packages into SQLite. |
| [tools/](../tools/README.md) | How to run the reference commands: `paperspec-check` checks everything, `paperspec-build` rebuilds the fixtures and vectors, and `paperspec-load` loads packages into SQLite. |
| [src/paperspec/](../src/paperspec/) | The oracle, a compact implementation of every rule, which the other repositories of the project install as the Python package `paperspec`. |
| [vectors/](../vectors/README.md) | 25 files of test vectors: inputs and the exact outputs every implementation must compute. |
| [fixtures/](../fixtures/README.md) | Five valid example packages, one of them also as a zip, 31 deliberately broken ones that each break one named rule, example records, and the test keys that signed them. All text is synthetic. |
@@ -73,8 +73,8 @@ The repository ships reference tools only for the checker and store roles.
| Reader | Opens a package, resolves addresses and anchors, and shows text, annotations and each passage's fidelity level. |
| Writer | Produces files in these formats: nodes, corrections, receipts, annotations, datasets. |
| Converter | Turns a witness (a PDF, JATS XML, the article format publishers use, a LaTeX source archive, an HTML page, an EPUB or a Typst project) into a package and raises doubtful readings as suspects (flagged disagreements awaiting review). |
| Checker | Recomputes hashes, validates files against their schemas and reports broken rules by rule ID. `tools/check.py` does this for the repository. |
| Store | Keeps packages and records in a database a team can clone. `tools/load.py` loads packages into SQLite. |
| Checker | Recomputes hashes, validates files against their schemas and reports broken rules by rule ID. `paperspec-check` does this for the repository. |
| Store | Keeps packages and records in a database a team can clone. `paperspec-load` loads packages into SQLite. |
The [specification's reading paths](../spec/README.md) say which chapters each role needs.
+2 -2
View File
@@ -13,7 +13,7 @@ It has nothing to do with the repository's name.
Open [fixtures/packages/hale2026harmonic-damb/](../fixtures/packages/hale2026harmonic-damb/) beside this page, because every ID below is in it.
A package is a folder of files, mostly JSON Lines (`.jsonl`: one JSON record per line).
The hand-written source of the first reading is [fixtures/src/hale2026harmonic.json](../fixtures/src/hale2026harmonic.json); no converter ran on it, and [`tools/build.py`](../tools/build.py) expands it into the package.
The hand-written source of the first reading is [fixtures/src/hale2026harmonic.json](../fixtures/src/hale2026harmonic.json); no converter ran on it, and [`paperspec-build`](../src/paperspec/cli/build.py) expands it into the package.
Each step says what a converter, a person or a checker does.
## 1. A PDF arrives
@@ -166,7 +166,7 @@ It is a [Frictionless Data Package](https://datapackage.org/standard/data-packag
Its CSV keeps every digit as printed (`1.526` and `1.5260` stay different), and every value has an anchor to the cell that prints it, so a claim that rests on a value can show the cell.
The descriptor types the fields as numbers; how a generic Data Package tool treats `1.5260` was not verified.
All twelve value anchors resolve as `current`.
The reference checker, [`tools/check.py`](../tools/check.py), confirms each CSV value equals its cell's text.
The reference checker, [`paperspec-check`](../src/paperspec/cli/check.py), confirms each CSV value equals its cell's text.
## 8. Sharing without the text