Publishing
- .gitea/workflows/publish.yml publishes paperspec to the instance's PyPI
registry (/api/packages/infra/pypi) when a v* tag is pushed: it checks
the repository, builds the sdist and wheel, refuses to upload unless the
built version equals the tag, runs twine check --strict, and uploads
with the secrets PACKAGE_USER and PACKAGE_TOKEN (--skip-existing, so a
rerun on the same tag is safe). Run by hand, it refuses anything but a
tag.
- pixi dev environment: twine >= 6.1 (the first that reads Metadata 2.4),
and `dist` (sdist and wheel) replaces the `wheel` task.
- deploy/README.md: how to set up the secrets, release a version, and
install from the registry with pixi, uv or pip, tying paperspec to the
registry where the installer allows it.
- CONTRIBUTING.md "Releasing": a tag versions the package; a format
change takes the new format version as the top CHANGELOG heading, and a
code-only release takes the next patch as a subsection, as 0.1.1 does.
- pyproject: fallback_version 0+unknown for source trees without git; the
license expression is Apache-2.0 AND CC0-1.0, for the code and the data
the wheel carries.
Cleanup (from an audit of every reference to the package)
- spec/00 lists src/ (the package) and tools/ (how to run it), and no
longer says the reference implementation is not a library.
- Stale explanations fixed in pixi.toml, .gitignore, the Caddyfile, the
vectors, context, sql and tools READMEs, docs/examples.md, docs/faq.md,
spec/01, docs/rationale.md and docs/comparison.md: the commands, what
they need (the tools extra), the fixture folder hale2026harmonic-damb,
and three lines that held two sentences.
- tools/README.md counts all six commands, says what needs which extra
(package.validate and corrections.gate need [schema]), and that the
vectors ship but the fixtures they name do not.
- The CLI's install and clone hints point at the registry instructions and
an https clone URL; schema.py raises an ImportError naming
paperspec[schema], as crypto.py does for [crypto].
- check's Unicode check now guards the package's pinned constant.
- site skips build/, dist/ and *.egg-info.
- CHANGELOG 0.1.1 describes the final state.
pixi run check passes (as `python -m paperspec check`); the rebuild is
byte-identical. pixi.lock needs `pixi lock` for twine before pushing: the
publish workflow installs with --locked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The six console scripts (paperspec-check, -build, -validate, -load,
-stress, -site) become subcommands of a single `paperspec` command:
paperspec validate, load, check, build, stress and site. `python -m
paperspec COMMAND` runs the same thing.
- paperspec/cli/main.py builds the whole parser, apart from the commands,
so `paperspec --help` and `paperspec COMMAND --help` work without a
checkout or the tools extra; a command's module is imported only when
it runs.
- `paperspec` with no command prints the overview: what each command does,
examples, and which work anywhere (validate, load) and which need a
checkout (check, build, stress, site). Each command's help has a
description with advice, and worked examples; a misspelled command gets
a suggestion (argparse suggest_on_error); `--version` prints the version
and the pinned Unicode version.
- Each command module exposes run(args) and returns the exit status.
Arguments that parse but do not fit together raise UsageError, which
prints the command's usage and exits 2: load now also refuses
--built-from or --store without --bundle.
- The failure messages say what to do: install paperspec[tools], clone the
repository, rerun stress with the printed seed, or run paperspec build
when the rebuild check differs. check and stress say how many failures
they did not list.
- check runs the rebuild as `python -m paperspec build`.
- pyproject installs one script, `paperspec`; the pixi tasks and the docs
use the subcommands.
pixi run check passes (run here as `python -m paperspec check` with src/ on
the path); the rebuild is byte-identical. pixi.lock still needs
`pixi lock`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Python and Unicode
- requires-python is >=3.14,<3.15: 3.14 is the only Python with the
Unicode 16.0 data the specification pins (3.13 has 15.1, 3.15 has 17.0).
- `import paperspec` raises ImportError unless unicodedata reports
paperspec.UNICODE_VERSION (16.0.0), so a source checkout or an install
that bypasses requires-python cannot compute values on other data.
Commands
- tools/{check,build,validate,load,stress,site}.py move with history to
src/paperspec/cli/ and install as paperspec-check, -build, -validate,
-load, -stress and -site; each also runs as `python -m paperspec.cli.X`.
- paperspec.cli.require() exits with "install paperspec[tools]" when
cryptography or jsonschema is missing; paperspec.cli.checkout() finds
the repository the check, build, stress and site commands work on (the
editable install's own checkout, else the one around the current
folder) and exits with a message outside one. tools/__init__.py is gone.
- validate and load read the schemas and SQL through the new
paperspec.resources.path(), which finds them in a wheel and in an
editable install alike, so they work anywhere the package is installed.
- check runs the rebuild as `python -m paperspec.cli.build`.
- build still writes the provenance tool label "paperspec tools/build.py":
it is part of every hashed and signed fixture.
- pixi tasks and the check workflow run the commands; tools/README.md
documents them, and the other docs name the commands instead of paths.
License
- LICENSE.md gives `src/` Apache-2.0 alongside `tools/`. Since the
previous commit moved the code to src/, no row had covered it.
pixi run check passes (run as `python -m paperspec.cli.check` with src/ on
the path, since this environment cannot install the entry points); the
byte-for-byte rebuild matches, so no fixture or vector changed.
pixi.lock still needs `pixi lock`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The oracle had two import names, `tools.oracle` for the tools and
`paperspec` for other repositories, which could load the same code twice
in one process and needed a gate keyed on the module name. It now lives
in the standard src layout under its one name.
- git mv tools/oracle -> src/paperspec; the 22 rule modules are unchanged.
- The tools import `paperspec`. `tools/__init__.py` now holds the Python
3.14 and cryptography/jsonschema check and puts src/ first on sys.path,
so the tools run this checkout's code with or without pixi. The
package's own __init__ checks nothing and exposes __version__.
- pyproject.toml: version from git tags via setuptools-scm
(guess-next-dev, node-and-timestamp, src/paperspec/_version.py,
gitignored). The packages stay listed rather than found because
schemas/, sql/ and vectors/ stay at the root: the spec links to them and
the site serves the schemas at their $id URLs.
- pixi.toml: a `dev` feature and environment with setuptools,
setuptools-scm and python-build, and `pixi run -e dev wheel`.
- check.py scans src/paperspec for rule IDs; README, CONTRIBUTING, the
FAQ, the overview, the vectors and tools READMEs and the changelog point
at the new path. The FAQ no longer says the oracle is not a library.
pixi.lock is not re-solved in this commit: with a dynamic version, pixi
builds the package metadata to lock, which needs the conda environment
installed. Run `pixi lock` and commit the result before pushing; the site
workflow installs with --locked.
pixi run check passes; no computed value, vector, fixture, schema or SQL
file changed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Other repositories (paperstack, then paperpack) import the reference
implementation instead of reimplementing the specification; scienting-core
will take the role later behind the same vectors.
- pyproject.toml: distribution and import name `paperspec`, version 0.1.0,
Python >= 3.12, no base dependencies; `tools/oracle` is the package, and
`schemas/`, `sql/` and `vectors/` ship as the data subpackages
`paperspec.schemas`, `paperspec.sql` and `paperspec.vectors`. Extras
`schema`, `crypto` and `tools`.
- tools/oracle/__init__.py: the Python 3.14 and cryptography/jsonschema gate
runs only when imported as `tools.oracle` by the repository's own tools.
- crypto.py imports `cryptography` on first use of HKDF and Ed25519, with an
ImportError naming `paperspec[crypto]`; schema.py imports `jsonschema` and
`referencing` in `Store.validator` and `Store.registry`. package.py, db.py
and receipts.py then import on a bare interpreter.
- pixi.toml installs the package editable (pixi.lock re-solved; no conda pin
changed), so the tools and the package resolve to the same code.
- tools/README.md names the library surface and the extras; CHANGELOG.md
records the packaging change under 0.1.0, since check.py requires the
changelog's first heading to equal the format version.
No computed value, vector, fixture, schema or SQL file changed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
openalex in same_as, else the folded title, year and family name. A
collision lengthens the later key by two characters; a key is minted
once and never recomputed. A document key adds .si, .v<seq> or .<type>
for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
every node and record ID was re-minted and the docs rewritten to match.
The validator checks the document key against the work key and version
(doc.key.work) and the hash part against the identifier (doc.key.suffix).
vectors/keys.jsonl pins the cases paperstack's tests pin.
Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
implemented in tools/oracle/vrf.py in plain Python and reproduced
against the RFC's Example 16; the sample ranks by the VRF output. The
earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
verify_inclusion, consistency_proof, verify_consistency) with
scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
checkpoints with key hashes as transparency logs share them, the rule
that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).
Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
the checker refuses another version.
Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
checkpoint records; a check that every ID and hash the prose quotes
exists in the fixtures or vectors; three more broken packages
(history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
section, media-type registration and OCI distribution notes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Rename manuspec to paperspec everywhere outside pixi.toml: prose, the
tools, provenance tool names and the fixture key derivations, so every
fixture and vector is rebuilt.
- Move the namespace addresses under the site, mirroring the repository:
schema $id https://paperspec.scient.ing/schemas/<file>, the JSON-LD
context https://paperspec.scient.ing/context/scienting-1.jsonld, and
term IRIs https://paperspec.scient.ing/vocab#<term>. Format
identifiers (scienting-*) and URNs (urn:scienting:) are unchanged.
- tools/site.py (standard library only) builds _site/: every file of the
repository at its own path, each Markdown file rendered as a page with
sidebar navigation, heading anchors matching the forge's, folder
listings, a vocabulary page for the context's IRIs, a heading search
and a 404 page; light and dark themes.
- check.py builds the site and checks that every Markdown file has a
page, every internal link and fragment resolves, no Markdown is left
unrendered, and every $id, the context address and the vocabulary IRIs
are files of the site.
- deploy/: a Caddyfile with the media types and CORS the data needs, and
a guide; .gitea/workflows/site.yml builds on main and publishes with
rsync when the SITE_* secrets are set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Add tools/validate.py: validates packages, and with --correction runs
proposed corrections through the gate against a package without
writing anything (spec/11 §11.4). The gate now refuses a proposal that
fails the correction schema, rule correction.schema.
- tools/load.py validates every package before loading, and with
--bundle --built-from builds a scienting-bundle/1 with the FTS5 search
index and bundle_meta (spec/13 §13.5). Search rows are defined per
unit (sentence, cell, equation outside a sentence, figure-text) with
the parent kind as context.
- Roster keys carry a role, and a key with the role owner may revoke any
receipt (spec/08 §8.7, spec/20); roster schema, store table and the
fixture roster updated. The checker signs a trial revocation to prove
the rule both ways.
- A qa: receipt raises the nodes it sampled except those in its errors
(spec/08 §8.2); the example paper's fidelity counts become 27 F3,
13 F2, 4 F1 and 1 F0.
- The destination chooses the keyed scheme, not the text policy:
hmac-team between the team's machines, hkdf-pages outside the team
(spec/05 §5.8, spec/12 §12.5).
- A TextQuoteSelector must repeat the anchor's plain quote and is absent
for hashed or keyed anchors (spec/18 §18.3).
- Add fixtures/records/suspect.json, a scienting-suspect/1 record.
- Oracle: hashes are kept incrementally and checked after every
correction against a full recomputation; load and write refuse unsafe
manifest paths; Ed25519, HKDF and schema validation come from the
cryptography and jsonschema packages; canonical JSON uses the built-in
encoder with an exact RFC 8785 fallback.
- The tools need Python 3.14 with cryptography and jsonschema; the py310
CI environment is dropped and the workflow runs the default only.
- Resolve every [CHECK] note in the docs, READMEs and context; add
CONTRIBUTING.md; every tool answers --help and refuses unknown options.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
alexmandClaude