Publish the package and its documentation, and move to the papers organization
Add the Zensical documentation of the Python package (docs/package, built by the docs workflow), merge the standard publish workflow with the setuptools-scm tag check, add the SPDX header rule to ruff, restructure the README, and point every repository and registry address at the Gitea organization papers. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
This commit is contained in:
37 files changed
+1005
-83
No files matched your search
@@ -0,0 +1,51 @@
|
||||
# Check and rebuild the repository
|
||||
|
||||
Four commands work on a checkout of the paperspec repository, not on an installed package: `check`, `build`, `stress` and `site`.
|
||||
The checkout is the one an editable install points at, or else the one around the current folder.
|
||||
|
||||
## Check everything
|
||||
|
||||
```sh
|
||||
paperspec check
|
||||
```
|
||||
|
||||
The checker validates every schema, record and package, confirms that every broken package is refused for the stated reason, recomputes every test vector, rebuilds everything and compares it byte for byte, loads every package into SQLite and back, and checks the prose, the links and the website.
|
||||
Each section prints how many checks it ran, and a passing run ends with `All checks pass.`
|
||||
The exit status is 0 when every check passed and 1 otherwise.
|
||||
|
||||
`paperspec check --quick` skips the byte-for-byte rebuild, the slowest part.
|
||||
The command does not run the Python code checks or the Markdown lint; `pixi run check` runs those first ([Install](../start/install.md#the-pixi-tasks)).
|
||||
|
||||
## Rebuild the vectors and fixtures
|
||||
|
||||
No expected value in the repository is typed by hand.
|
||||
The vectors and fixtures are computed from the hand-written sources in `fixtures/src/` and the cases the builder defines in `src/paperspec/cli/build.py`.
|
||||
After you change a rule in the code, a fixture source or a case, rebuild and commit the regenerated files with the change:
|
||||
|
||||
```sh
|
||||
paperspec build
|
||||
```
|
||||
|
||||
`paperspec build --out DIR` builds elsewhere, so you can compare with `diff -r DIR/vectors vectors` before overwriting anything.
|
||||
The build is deterministic, and `paperspec check` proves it by rebuilding into a temporary folder and comparing every file.
|
||||
|
||||
## Stress the oracle
|
||||
|
||||
```sh
|
||||
paperspec stress --rounds 5
|
||||
```
|
||||
|
||||
The vectors pin values; `stress` checks the invariants the chapters state, such as canonical JSON round trips and the zip reader never writing outside its folder, on random inputs from a seeded generator.
|
||||
A failure prints its seed, so `paperspec stress --seed SEED` reproduces it.
|
||||
The defaults are the seed 20261014 and 200 cases per property.
|
||||
Before a release, run it with the default seed and with one or two others.
|
||||
|
||||
## Build the website
|
||||
|
||||
```sh
|
||||
paperspec site --out /tmp/paperspec-site
|
||||
```
|
||||
|
||||
The site mirrors the repository: each file is served at its own path, each Markdown file gets a rendered page beside it, and the schemas are at the addresses in their `$id`.
|
||||
It needs only the standard library.
|
||||
The pages you are reading are a separate site, built with Zensical by `pixi run -e docs docs-build`.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Guides
|
||||
|
||||
Each guide takes one task from start to finish, with commands that were run against the repository's synthetic fixtures.
|
||||
|
||||
| Guide | For |
|
||||
| --- | --- |
|
||||
| [Validate a package](validate-a-package.md) | Check a package folder or a zipped package against every rule, and try a proposed correction without writing it. |
|
||||
| [Load packages into SQLite](load-into-sqlite.md) | Put packages in a database you can query, or build a bundle with full-text search. |
|
||||
| [Use the library](use-the-library.md) | Compute hashes and identifiers, read a package and run the gate from Python. |
|
||||
| [Check and rebuild the repository](check-and-build.md) | Run the repository's checks, regenerate the vectors and fixtures, stress the oracle and build the website. |
|
||||
|
||||
The formats themselves are explained in the guides at <https://paperspec.scient.ing/docs/>.
|
||||
@@ -0,0 +1,46 @@
|
||||
# Load packages into SQLite
|
||||
|
||||
`paperspec load` validates packages and loads them into a SQLite database with the tables of `sql/doc.sql` and `sql/store.sql`.
|
||||
A node, a reference, a correction and a receipt each get rows you can join and query.
|
||||
|
||||
## Load a package
|
||||
|
||||
```sh
|
||||
paperspec load --sqlite lab.db fixtures/packages/hale2026harmonic-damb
|
||||
```
|
||||
|
||||
```text
|
||||
hale2026harmonic-damb: 75 nodes
|
||||
```
|
||||
|
||||
- Every package is validated first, and nothing is loaded if any fails.
|
||||
Each failure is printed as `package: where: rule: message`, as `paperspec validate` prints it.
|
||||
- A new file gets the tables, and an existing one takes more packages, so you can add papers to the same database later.
|
||||
- Files with no table of their own (assets, datasets, derived files) are copied by SHA-256 into a folder beside the database, `lab.db.assets/`, which must travel with it.
|
||||
- The full and checks-only forms of one document share its key, so load them into separate databases.
|
||||
|
||||
## Query it
|
||||
|
||||
```sh
|
||||
python -m sqlite3 lab.db "SELECT kind, COUNT(*) FROM nodes GROUP BY kind ORDER BY 2 DESC, kind LIMIT 3"
|
||||
```
|
||||
|
||||
```text
|
||||
('cell', 23)
|
||||
('sentence', 18)
|
||||
('row', 6)
|
||||
```
|
||||
|
||||
The guide at <https://paperspec.scient.ing/docs/examples.html> has queries to start from: the sentences still below fidelity F2, the citation graph at sentence level, and who signed what.
|
||||
|
||||
## Build a bundle with full-text search
|
||||
|
||||
```sh
|
||||
paperspec load --sqlite search.db --bundle --built-from REV paper/ other/
|
||||
```
|
||||
|
||||
A bundle is a `scienting-bundle/1` file: it also gets the tables of `sql/bundle-sqlite.sql`, the full-text search index and `bundle_meta`.
|
||||
It is built once, into a file that does not exist yet.
|
||||
`--bundle` needs `--built-from`, which names the store revision the packages come from, and the command prints the number of search rows it added.
|
||||
`--store` names the store, and defaults to the packages' common store.
|
||||
`--built-from` and `--store` are refused without `--bundle`.
|
||||
@@ -0,0 +1,66 @@
|
||||
# Use the library
|
||||
|
||||
The modules of `paperspec` compute every value the formats define, so a program that writes or reads packages can import them instead of reimplementing the specification.
|
||||
Every module imports with the standard library alone.
|
||||
Schema validation needs `paperspec[schema]`, and Ed25519 and HKDF need `paperspec[crypto]`; `paperspec[tools]` installs both ([Install](../start/install.md)).
|
||||
|
||||
## Canonical JSON and hashes
|
||||
|
||||
`jcs.dumps` writes the canonical JSON the formats hash and store: sorted keys, no spaces, and numbers written as an ECMAScript engine would.
|
||||
`text.sha256_hex` hashes a string, and `text.norm` applies the normalization the five hashes start from.
|
||||
|
||||
```python
|
||||
from paperspec import jcs, text
|
||||
|
||||
print(jcs.dumps({"b": 1.0, "a": [2, "é"]}))
|
||||
print(text.norm("a b\n c"))
|
||||
print(text.sha256_hex("The two parameter sets agree to within 2% for every bond."))
|
||||
```
|
||||
|
||||
```text
|
||||
{"a":[2,"é"],"b":1}
|
||||
a b c
|
||||
175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a
|
||||
```
|
||||
|
||||
## Validate a package
|
||||
|
||||
`package.validate` checks a package folder against every rule and returns a report whose `errors` list is empty when the package is valid.
|
||||
The schema store reads the schemas from wherever the package carries them, in an installed wheel or at the root of a checkout.
|
||||
|
||||
```python
|
||||
from paperspec import package, resources
|
||||
from paperspec.schema import Store
|
||||
|
||||
store = Store(resources.path("schemas"))
|
||||
report = package.validate("fixtures/packages/hale2026harmonic-damb", store)
|
||||
print(bool(report), report.errors)
|
||||
```
|
||||
|
||||
```text
|
||||
True []
|
||||
```
|
||||
|
||||
Each entry of `errors` is a dictionary with `rule`, `where` and `message`, the three fields `paperspec validate` prints.
|
||||
|
||||
## Read a package and run the gate
|
||||
|
||||
`package.read_state` reads a package folder into a state that holds its nodes, and `corrections.gate` runs a proposed correction against it without changing it.
|
||||
It returns the state the correction would produce and the record it would append, or raises `tree.Refusal`, which names the rule broken.
|
||||
|
||||
```python
|
||||
from paperspec import corrections, package, resources
|
||||
from paperspec.schema import Store
|
||||
|
||||
state = package.read_state("fixtures/packages/tiny2026tiny-j4qm")
|
||||
store = Store(resources.path("schemas"))
|
||||
new_state, record = corrections.gate(state, proposal, store)
|
||||
```
|
||||
|
||||
Here `proposal` is a parsed correction such as the one in [Validate a package](validate-a-package.md).
|
||||
|
||||
## Check your own code against the vectors
|
||||
|
||||
The vectors travel with the package as the data subpackage `paperspec.vectors`, and `resources.path("vectors")` finds the folder in an installed wheel and in an editable install alike.
|
||||
Four of them, `resolve.jsonl`, `grading.jsonl`, `targets.jsonl` and `traces.jsonl`, name a fixture package, and the fixtures stay in the repository.
|
||||
The [reference](../reference/api.md) lists the modules.
|
||||
@@ -0,0 +1,43 @@
|
||||
# Validate a package
|
||||
|
||||
A package is a folder, or a zipped package (`.sct.zip`), that holds one document.
|
||||
`paperspec validate` checks it against every rule of the specification's conformance chapter and works anywhere the package is installed with its `tools` extra.
|
||||
|
||||
## Check one or several packages
|
||||
|
||||
```sh
|
||||
paperspec validate fixtures/packages/hale2026harmonic-damb
|
||||
paperspec validate fixtures/packages/tiny2026tiny-j4qm.sct.zip fixtures/packages/tiny2026tiny-j4qm
|
||||
```
|
||||
|
||||
Each valid package prints `PACKAGE: valid` and the exit status is 0.
|
||||
A failure goes to standard error as `package: where: rule: message`, the exit status is 1, and nothing else prints.
|
||||
A usage error, such as no package at all, exits with status 2.
|
||||
|
||||
A zipped package is unpacked under the specification's path rules before anything else happens: a path that leaves the folder, a symbolic link and an oversize archive are refused.
|
||||
`--max-size BYTES` sets the largest size an archive may expand to; the default is 1 GiB.
|
||||
|
||||
## Read a refusal
|
||||
|
||||
The rule ID names the rule that failed.
|
||||
`https://paperspec.scient.ing/registry/rules.json` lists every ID with the chapter of the specification that states it, which is where to read why.
|
||||
The folder `fixtures/invalid/` holds one package for each of many rules; its `expect.json` says which rules the package must trigger.
|
||||
|
||||
## Try a correction before making it
|
||||
|
||||
A correction changes one node of a document and records why.
|
||||
Before a tool writes one, it can run the proposal through the gate, which needs only the package's current state:
|
||||
|
||||
```sh
|
||||
paperspec validate --correction fix.json fixtures/packages/tiny2026tiny-j4qm
|
||||
```
|
||||
|
||||
`fix.json` holds one proposal as a JSON object, or several as JSON Lines, applied in order:
|
||||
|
||||
```json
|
||||
{"format":"scienting-correction/1","op":"emend","target":"j663d4h","find":"second","replace":"next","kind":"wording","status":"proposed","evidence":[{"kind":"person","by":"ana","note":"The editor prefers a plainer word."}],"author":{"key":"ana"},"reason":"The authors misuse the ordinal; the paragraph follows the first.","at":"2026-10-03T10:00:00Z"}
|
||||
```
|
||||
|
||||
The command prints the record the correction would append to `corrections.jsonl`, with its computed `index`, `id`, `path`, `before` and `after`, and writes nothing.
|
||||
A refused proposal prints `file:N node: rule: message` on standard error and stops: with this proposal, changing `second` to `2nd` is refused with `correction.digits`, because an emendation changes no digit.
|
||||
`--correction` takes exactly one package, and it refuses a package that withholds its text.
|
||||
Reference in new issue
Block a user