A data body says what kind of data a table, figure panel or passage presents:
one of fifteen kinds, the dataset that holds the values, what each field
measures (with the dataset's own unit) and the molecules and sequences it is
about, by the identities of structure and sequence bodies. Where the values
came from stays the dataset's origin.
- spec/18 section 18.8, the body in annotation.schema.json, 'data' in the
store's body_kind CHECK, five terms in the JSON-LD context.
- paperspec.data.problems and the rules layer.data.type, layer.data.quantity,
layer.data.subject and layer.data.dataset. Units are compared as strings
with the dataset's; no unit library is needed.
- fixtures/records/data/ (three annotations on the main example) and
vectors/data.jsonl (35 cases); paperspec check covers both, and the
determinism rebuild matches.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
Chemical structures (spec/18 section 18.6) and biological sequences (18.7) tied to
the words of a paper. Minor change: two new body kinds, optional properties and
seven rules, with no valid file made invalid.
- annotation schema: structure (SMILES or SMARTS, role, identity, written, label,
accessions, force-field terms) and sequence (alphabet, role, identity, start,
organism, regions, variants); body_kind in sql/store.sql; nine terms in the
JSON-LD context; rules layer.structure.{value,identity,term} and
layer.sequence.{value,identity,region,variant}.
- paperspec.structures reads the atoms and bonds of a SMILES or SMARTS with the
standard library, so a term's atoms are checked without a toolkit; checked against
RDKit on a corpus and random texts. paperspec.sequences checks alphabets,
identities, regions and variants.
- fixtures/records/chemistry (on the main example) and biology (on the draft, which
gains a last sentence), vectors/structures.jsonl and vectors/sequences.jsonl;
check.py runs both layers and the vectors; counts, guides and CHANGELOG updated.
- the prose check no longer takes organism for a UK spelling.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
A trace follows one citation of a sentence of a citing paper to a passage of the
cited paper (spec/18 section 18.5). Minor change: a new body kind, optional
properties and two rules, with no valid file made invalid.
- annotation schema branch (source, selector, reference, note, verdict, proposer;
proposer required for a Software creator); body_kind in sql/store.sql; the note
and proposer terms of the JSON-LD context; rules layer.trace.source and
layer.trace.target.
- paperspec.traces: resolve(annotation, citing_state, cited_state, same_as) gives
current, changed, stale, unlinked or misresolved with open and cites; problems,
disputes, disputed.
- fixtures/records/traces/ on writer2027fixed-ynr7.v3 and vectors/traces.jsonl
(29 cases); check.py runs the record checks over both layers; counts, glossary,
examples and CHANGELOG updated.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uy4ZYbWRoCdUtWLEfigysY
Math content may say its source is in scienting-math/1's canonical TeX (form,
in no hash); derived/math.jsonl holds papermath's tree, symbols and issues,
and the validator now checks derived files. A symbol annotation names its
symbol by key; lode claim references use lode's tag and pin syntax. The zip
reader refuses entries zipfile cannot read, and checkpoint lines end in LF only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017R7sgDNvUGWY2zFPCq2JH6
scienting-layer/1: an optional, fixed category on comment, caveat and
question bodies (comment, context, critique, question, correction,
feedback, resource).
Corrections: emend fixes the authors' own slip and keeps what was printed
(a run's printed), never lowering fidelity or a witness match; relink sets
or removes what a citation names; resolve sets a reference's resolution.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017R7sgDNvUGWY2zFPCq2JH6
Publishing
- .gitea/workflows/publish.yml publishes paperspec to the instance's PyPI
registry (/api/packages/infra/pypi) when a v* tag is pushed: it checks
the repository, builds the sdist and wheel, refuses to upload unless the
built version equals the tag, runs twine check --strict, and uploads
with the secrets PACKAGE_USER and PACKAGE_TOKEN (--skip-existing, so a
rerun on the same tag is safe). Run by hand, it refuses anything but a
tag.
- pixi dev environment: twine >= 6.1 (the first that reads Metadata 2.4),
and `dist` (sdist and wheel) replaces the `wheel` task.
- deploy/README.md: how to set up the secrets, release a version, and
install from the registry with pixi, uv or pip, tying paperspec to the
registry where the installer allows it.
- CONTRIBUTING.md "Releasing": a tag versions the package; a format
change takes the new format version as the top CHANGELOG heading, and a
code-only release takes the next patch as a subsection, as 0.1.1 does.
- pyproject: fallback_version 0+unknown for source trees without git; the
license expression is Apache-2.0 AND CC0-1.0, for the code and the data
the wheel carries.
Cleanup (from an audit of every reference to the package)
- spec/00 lists src/ (the package) and tools/ (how to run it), and no
longer says the reference implementation is not a library.
- Stale explanations fixed in pixi.toml, .gitignore, the Caddyfile, the
vectors, context, sql and tools READMEs, docs/examples.md, docs/faq.md,
spec/01, docs/rationale.md and docs/comparison.md: the commands, what
they need (the tools extra), the fixture folder hale2026harmonic-damb,
and three lines that held two sentences.
- tools/README.md counts all six commands, says what needs which extra
(package.validate and corrections.gate need [schema]), and that the
vectors ship but the fixtures they name do not.
- The CLI's install and clone hints point at the registry instructions and
an https clone URL; schema.py raises an ImportError naming
paperspec[schema], as crypto.py does for [crypto].
- check's Unicode check now guards the package's pinned constant.
- site skips build/, dist/ and *.egg-info.
- CHANGELOG 0.1.1 describes the final state.
pixi run check passes (as `python -m paperspec check`); the rebuild is
byte-identical. pixi.lock needs `pixi lock` for twine before pushing: the
publish workflow installs with --locked.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The six console scripts (paperspec-check, -build, -validate, -load,
-stress, -site) become subcommands of a single `paperspec` command:
paperspec validate, load, check, build, stress and site. `python -m
paperspec COMMAND` runs the same thing.
- paperspec/cli/main.py builds the whole parser, apart from the commands,
so `paperspec --help` and `paperspec COMMAND --help` work without a
checkout or the tools extra; a command's module is imported only when
it runs.
- `paperspec` with no command prints the overview: what each command does,
examples, and which work anywhere (validate, load) and which need a
checkout (check, build, stress, site). Each command's help has a
description with advice, and worked examples; a misspelled command gets
a suggestion (argparse suggest_on_error); `--version` prints the version
and the pinned Unicode version.
- Each command module exposes run(args) and returns the exit status.
Arguments that parse but do not fit together raise UsageError, which
prints the command's usage and exits 2: load now also refuses
--built-from or --store without --bundle.
- The failure messages say what to do: install paperspec[tools], clone the
repository, rerun stress with the printed seed, or run paperspec build
when the rebuild check differs. check and stress say how many failures
they did not list.
- check runs the rebuild as `python -m paperspec build`.
- pyproject installs one script, `paperspec`; the pixi tasks and the docs
use the subcommands.
pixi run check passes (run here as `python -m paperspec check` with src/ on
the path); the rebuild is byte-identical. pixi.lock still needs
`pixi lock`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Python and Unicode
- requires-python is >=3.14,<3.15: 3.14 is the only Python with the
Unicode 16.0 data the specification pins (3.13 has 15.1, 3.15 has 17.0).
- `import paperspec` raises ImportError unless unicodedata reports
paperspec.UNICODE_VERSION (16.0.0), so a source checkout or an install
that bypasses requires-python cannot compute values on other data.
Commands
- tools/{check,build,validate,load,stress,site}.py move with history to
src/paperspec/cli/ and install as paperspec-check, -build, -validate,
-load, -stress and -site; each also runs as `python -m paperspec.cli.X`.
- paperspec.cli.require() exits with "install paperspec[tools]" when
cryptography or jsonschema is missing; paperspec.cli.checkout() finds
the repository the check, build, stress and site commands work on (the
editable install's own checkout, else the one around the current
folder) and exits with a message outside one. tools/__init__.py is gone.
- validate and load read the schemas and SQL through the new
paperspec.resources.path(), which finds them in a wheel and in an
editable install alike, so they work anywhere the package is installed.
- check runs the rebuild as `python -m paperspec.cli.build`.
- build still writes the provenance tool label "paperspec tools/build.py":
it is part of every hashed and signed fixture.
- pixi tasks and the check workflow run the commands; tools/README.md
documents them, and the other docs name the commands instead of paths.
License
- LICENSE.md gives `src/` Apache-2.0 alongside `tools/`. Since the
previous commit moved the code to src/, no row had covered it.
pixi run check passes (run as `python -m paperspec.cli.check` with src/ on
the path, since this environment cannot install the entry points); the
byte-for-byte rebuild matches, so no fixture or vector changed.
pixi.lock still needs `pixi lock`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
openalex in same_as, else the folded title, year and family name. A
collision lengthens the later key by two characters; a key is minted
once and never recomputed. A document key adds .si, .v<seq> or .<type>
for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
every node and record ID was re-minted and the docs rewritten to match.
The validator checks the document key against the work key and version
(doc.key.work) and the hash part against the identifier (doc.key.suffix).
vectors/keys.jsonl pins the cases paperstack's tests pin.
Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
implemented in tools/oracle/vrf.py in plain Python and reproduced
against the RFC's Example 16; the sample ranks by the VRF output. The
earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
verify_inclusion, consistency_proof, verify_consistency) with
scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
checkpoints with key hashes as transparency logs share them, the rule
that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).
Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
the checker refuses another version.
Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
checkpoint records; a check that every ID and hash the prose quotes
exists in the fixtures or vectors; three more broken packages
(history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
section, media-type registration and OCI distribution notes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Rename manuspec to paperspec everywhere outside pixi.toml: prose, the
tools, provenance tool names and the fixture key derivations, so every
fixture and vector is rebuilt.
- Move the namespace addresses under the site, mirroring the repository:
schema $id https://paperspec.scient.ing/schemas/<file>, the JSON-LD
context https://paperspec.scient.ing/context/scienting-1.jsonld, and
term IRIs https://paperspec.scient.ing/vocab#<term>. Format
identifiers (scienting-*) and URNs (urn:scienting:) are unchanged.
- tools/site.py (standard library only) builds _site/: every file of the
repository at its own path, each Markdown file rendered as a page with
sidebar navigation, heading anchors matching the forge's, folder
listings, a vocabulary page for the context's IRIs, a heading search
and a 404 page; light and dark themes.
- check.py builds the site and checks that every Markdown file has a
page, every internal link and fragment resolves, no Markdown is left
unrendered, and every $id, the context address and the vocabulary IRIs
are files of the site.
- deploy/: a Caddyfile with the media types and CORS the data needs, and
a guide; .gitea/workflows/site.yml builds on main and publishes with
rsync when the SITE_* secrets are set.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Add tools/validate.py: validates packages, and with --correction runs
proposed corrections through the gate against a package without
writing anything (spec/11 §11.4). The gate now refuses a proposal that
fails the correction schema, rule correction.schema.
- tools/load.py validates every package before loading, and with
--bundle --built-from builds a scienting-bundle/1 with the FTS5 search
index and bundle_meta (spec/13 §13.5). Search rows are defined per
unit (sentence, cell, equation outside a sentence, figure-text) with
the parent kind as context.
- Roster keys carry a role, and a key with the role owner may revoke any
receipt (spec/08 §8.7, spec/20); roster schema, store table and the
fixture roster updated. The checker signs a trial revocation to prove
the rule both ways.
- A qa: receipt raises the nodes it sampled except those in its errors
(spec/08 §8.2); the example paper's fidelity counts become 27 F3,
13 F2, 4 F1 and 1 F0.
- The destination chooses the keyed scheme, not the text policy:
hmac-team between the team's machines, hkdf-pages outside the team
(spec/05 §5.8, spec/12 §12.5).
- A TextQuoteSelector must repeat the anchor's plain quote and is absent
for hashed or keyed anchors (spec/18 §18.3).
- Add fixtures/records/suspect.json, a scienting-suspect/1 record.
- Oracle: hashes are kept incrementally and checked after every
correction against a full recomputation; load and write refuse unsafe
manifest paths; Ed25519, HKDF and schema validation come from the
cryptography and jsonschema packages; canonical JSON uses the built-in
encoder with an exact RFC 8785 fallback.
- The tools need Python 3.14 with cryptography and jsonschema; the py310
CI environment is dropped and the workflow runs the default only.
- Resolve every [CHECK] note in the docs, READMEs and context; add
CONTRIBUTING.md; every tool answers --help and refuses unknown options.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>