Publish the package and its documentation, and move to the papers organization

Add the Zensical documentation of the Python package (docs/package, built by the docs
workflow), merge the standard publish workflow with the setuptools-scm tag check, add the
SPDX header rule to ruff, restructure the README, and point every repository and registry
address at the Gitea organization papers.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
This commit is contained in:
alexmandClaude committed 2026-10-19 04:23:49 -04:00
1 parent bdbf03ebee
commit 30d310f64d
37 files changed
+1005 -83

No files matched your search

+51
View File
@@ -0,0 +1,51 @@
# Check and rebuild the repository
Four commands work on a checkout of the paperspec repository, not on an installed package: `check`, `build`, `stress` and `site`.
The checkout is the one an editable install points at, or else the one around the current folder.
## Check everything
```sh
paperspec check
```
The checker validates every schema, record and package, confirms that every broken package is refused for the stated reason, recomputes every test vector, rebuilds everything and compares it byte for byte, loads every package into SQLite and back, and checks the prose, the links and the website.
Each section prints how many checks it ran, and a passing run ends with `All checks pass.`
The exit status is 0 when every check passed and 1 otherwise.
`paperspec check --quick` skips the byte-for-byte rebuild, the slowest part.
The command does not run the Python code checks or the Markdown lint; `pixi run check` runs those first ([Install](../start/install.md#the-pixi-tasks)).
## Rebuild the vectors and fixtures
No expected value in the repository is typed by hand.
The vectors and fixtures are computed from the hand-written sources in `fixtures/src/` and the cases the builder defines in `src/paperspec/cli/build.py`.
After you change a rule in the code, a fixture source or a case, rebuild and commit the regenerated files with the change:
```sh
paperspec build
```
`paperspec build --out DIR` builds elsewhere, so you can compare with `diff -r DIR/vectors vectors` before overwriting anything.
The build is deterministic, and `paperspec check` proves it by rebuilding into a temporary folder and comparing every file.
## Stress the oracle
```sh
paperspec stress --rounds 5
```
The vectors pin values; `stress` checks the invariants the chapters state, such as canonical JSON round trips and the zip reader never writing outside its folder, on random inputs from a seeded generator.
A failure prints its seed, so `paperspec stress --seed SEED` reproduces it.
The defaults are the seed 20261014 and 200 cases per property.
Before a release, run it with the default seed and with one or two others.
## Build the website
```sh
paperspec site --out /tmp/paperspec-site
```
The site mirrors the repository: each file is served at its own path, each Markdown file gets a rendered page beside it, and the schemas are at the addresses in their `$id`.
It needs only the standard library.
The pages you are reading are a separate site, built with Zensical by `pixi run -e docs docs-build`.
+12
View File
@@ -0,0 +1,12 @@
# Guides
Each guide takes one task from start to finish, with commands that were run against the repository's synthetic fixtures.
| Guide | For |
| --- | --- |
| [Validate a package](validate-a-package.md) | Check a package folder or a zipped package against every rule, and try a proposed correction without writing it. |
| [Load packages into SQLite](load-into-sqlite.md) | Put packages in a database you can query, or build a bundle with full-text search. |
| [Use the library](use-the-library.md) | Compute hashes and identifiers, read a package and run the gate from Python. |
| [Check and rebuild the repository](check-and-build.md) | Run the repository's checks, regenerate the vectors and fixtures, stress the oracle and build the website. |
The formats themselves are explained in the guides at <https://paperspec.scient.ing/docs/>.
+46
View File
@@ -0,0 +1,46 @@
# Load packages into SQLite
`paperspec load` validates packages and loads them into a SQLite database with the tables of `sql/doc.sql` and `sql/store.sql`.
A node, a reference, a correction and a receipt each get rows you can join and query.
## Load a package
```sh
paperspec load --sqlite lab.db fixtures/packages/hale2026harmonic-damb
```
```text
hale2026harmonic-damb: 75 nodes
```
- Every package is validated first, and nothing is loaded if any fails.
Each failure is printed as `package: where: rule: message`, as `paperspec validate` prints it.
- A new file gets the tables, and an existing one takes more packages, so you can add papers to the same database later.
- Files with no table of their own (assets, datasets, derived files) are copied by SHA-256 into a folder beside the database, `lab.db.assets/`, which must travel with it.
- The full and checks-only forms of one document share its key, so load them into separate databases.
## Query it
```sh
python -m sqlite3 lab.db "SELECT kind, COUNT(*) FROM nodes GROUP BY kind ORDER BY 2 DESC, kind LIMIT 3"
```
```text
('cell', 23)
('sentence', 18)
('row', 6)
```
The guide at <https://paperspec.scient.ing/docs/examples.html> has queries to start from: the sentences still below fidelity F2, the citation graph at sentence level, and who signed what.
## Build a bundle with full-text search
```sh
paperspec load --sqlite search.db --bundle --built-from REV paper/ other/
```
A bundle is a `scienting-bundle/1` file: it also gets the tables of `sql/bundle-sqlite.sql`, the full-text search index and `bundle_meta`.
It is built once, into a file that does not exist yet.
`--bundle` needs `--built-from`, which names the store revision the packages come from, and the command prints the number of search rows it added.
`--store` names the store, and defaults to the packages' common store.
`--built-from` and `--store` are refused without `--bundle`.
+66
View File
@@ -0,0 +1,66 @@
# Use the library
The modules of `paperspec` compute every value the formats define, so a program that writes or reads packages can import them instead of reimplementing the specification.
Every module imports with the standard library alone.
Schema validation needs `paperspec[schema]`, and Ed25519 and HKDF need `paperspec[crypto]`; `paperspec[tools]` installs both ([Install](../start/install.md)).
## Canonical JSON and hashes
`jcs.dumps` writes the canonical JSON the formats hash and store: sorted keys, no spaces, and numbers written as an ECMAScript engine would.
`text.sha256_hex` hashes a string, and `text.norm` applies the normalization the five hashes start from.
```python
from paperspec import jcs, text
print(jcs.dumps({"b": 1.0, "a": [2, "é"]}))
print(text.norm("a b\n c"))
print(text.sha256_hex("The two parameter sets agree to within 2% for every bond."))
```
```text
{"a":[2,"é"],"b":1}
a b c
175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a
```
## Validate a package
`package.validate` checks a package folder against every rule and returns a report whose `errors` list is empty when the package is valid.
The schema store reads the schemas from wherever the package carries them, in an installed wheel or at the root of a checkout.
```python
from paperspec import package, resources
from paperspec.schema import Store
store = Store(resources.path("schemas"))
report = package.validate("fixtures/packages/hale2026harmonic-damb", store)
print(bool(report), report.errors)
```
```text
True []
```
Each entry of `errors` is a dictionary with `rule`, `where` and `message`, the three fields `paperspec validate` prints.
## Read a package and run the gate
`package.read_state` reads a package folder into a state that holds its nodes, and `corrections.gate` runs a proposed correction against it without changing it.
It returns the state the correction would produce and the record it would append, or raises `tree.Refusal`, which names the rule broken.
```python
from paperspec import corrections, package, resources
from paperspec.schema import Store
state = package.read_state("fixtures/packages/tiny2026tiny-j4qm")
store = Store(resources.path("schemas"))
new_state, record = corrections.gate(state, proposal, store)
```
Here `proposal` is a parsed correction such as the one in [Validate a package](validate-a-package.md).
## Check your own code against the vectors
The vectors travel with the package as the data subpackage `paperspec.vectors`, and `resources.path("vectors")` finds the folder in an installed wheel and in an editable install alike.
Four of them, `resolve.jsonl`, `grading.jsonl`, `targets.jsonl` and `traces.jsonl`, name a fixture package, and the fixtures stay in the repository.
The [reference](../reference/api.md) lists the modules.
+43
View File
@@ -0,0 +1,43 @@
# Validate a package
A package is a folder, or a zipped package (`.sct.zip`), that holds one document.
`paperspec validate` checks it against every rule of the specification's conformance chapter and works anywhere the package is installed with its `tools` extra.
## Check one or several packages
```sh
paperspec validate fixtures/packages/hale2026harmonic-damb
paperspec validate fixtures/packages/tiny2026tiny-j4qm.sct.zip fixtures/packages/tiny2026tiny-j4qm
```
Each valid package prints `PACKAGE: valid` and the exit status is 0.
A failure goes to standard error as `package: where: rule: message`, the exit status is 1, and nothing else prints.
A usage error, such as no package at all, exits with status 2.
A zipped package is unpacked under the specification's path rules before anything else happens: a path that leaves the folder, a symbolic link and an oversize archive are refused.
`--max-size BYTES` sets the largest size an archive may expand to; the default is 1 GiB.
## Read a refusal
The rule ID names the rule that failed.
`https://paperspec.scient.ing/registry/rules.json` lists every ID with the chapter of the specification that states it, which is where to read why.
The folder `fixtures/invalid/` holds one package for each of many rules; its `expect.json` says which rules the package must trigger.
## Try a correction before making it
A correction changes one node of a document and records why.
Before a tool writes one, it can run the proposal through the gate, which needs only the package's current state:
```sh
paperspec validate --correction fix.json fixtures/packages/tiny2026tiny-j4qm
```
`fix.json` holds one proposal as a JSON object, or several as JSON Lines, applied in order:
```json
{"format":"scienting-correction/1","op":"emend","target":"j663d4h","find":"second","replace":"next","kind":"wording","status":"proposed","evidence":[{"kind":"person","by":"ana","note":"The editor prefers a plainer word."}],"author":{"key":"ana"},"reason":"The authors misuse the ordinal; the paragraph follows the first.","at":"2026-10-03T10:00:00Z"}
```
The command prints the record the correction would append to `corrections.jsonl`, with its computed `index`, `id`, `path`, `before` and `after`, and writes nothing.
A refused proposal prints `file:N node: rule: message` on standard error and stops: with this proposal, changing `second` to `2nd` is refused with `correction.digits`, because an emendation changes no digit.
`--correction` takes exactly one package, and it refuses a package that withholds its text.