paperspec/CHANGELOG.md

Changelog

A change to any computed value is a breaking change and needs a new major version of the format it touches; see spec/00-introduction.md.

0.1.0

The first release.

• The specification, in chapters 0 to 23, covering every format: 32 formats, of which 19 are candidate and 13 provisional.
• 42 JSON Schemas, using a keyword subset that independent validators read alike.
• Tables for SQLite, Dolt and PostgreSQL, a team's store, and the SQLite bundle with full-text search.
• The JSON-LD context for layers, and registries of formats, rules, download-stamp rules and default stage requirements.
• Test vectors for every computed value, from canonical JSON to anchor resolution, grading and the target set of targeted F3.
• Example packages (a converted chemistry paper with corrections and receipts, its checks-only export, an open-access paper, a Typst draft and a tiny document, also as a zip), a record of every other format, and 31 broken packages, each breaking one named rule.
• Work keys minted from a Scholar-style citation key and a hash of the work's identity, with document keys for versions; tag: URIs for layers, annotations and documents.
• Spot-check seeds as VRF proofs (RFC 9381), RFC 9162 inclusion and consistency proofs, a history root over every document's corrections, and a provisional transparency log with signed checkpoints.
• Signed QA reports, a conformance checklist covered by a receipt, and a description annotation for the figure descriptions an accessible export needs.
• ML-DSA-65 (FIPS 204) as a signature scheme, with a reference implementation checked against NIST's vectors; cose reserved; did:key identifiers for roster keys; the hash algorithm named in every document; model provenance by weights, prompt and parameters; witnesses and a static tile layout for the log; and the character data pinned to Unicode 16.0.
• Guides for newcomers: an overview, the concepts, a walkthrough of the example paper, runnable examples, the rationale, a FAQ and a glossary.
• Tools that build and check everything, validate packages and zipped packages and run proposed corrections through the gate, load packages into SQLite or a bundle, and stress the reference implementation with random inputs.
• A pixi workspace pinned to Python 3.14, with tasks for checking, building, linting, validating and loading.

0.1.1

Packaging, commands and lazy imports only; the formats stay at 0.1.0, and no computed value, vector, fixture, schema or SQL file changed.

• The oracle moves from tools/oracle/ to src/paperspec/ and installs as the Python package paperspec (pyproject.toml, setuptools), with schemas/, sql/ and vectors/ as the data subpackages paperspec.schemas, paperspec.sql and paperspec.vectors, so other repositories import the reference implementation instead of reimplementing it.
• The package version comes from git tags through setuptools-scm.
• The base package has no dependencies and requires Python 3.14, the only version with the Unicode 16.0 character data the specification pins, and refuses to import under any other Unicode version; the extras paperspec[schema] (jsonschema, referencing), paperspec[crypto] (cryptography) and paperspec[tools] (both) cover schema and the HKDF and Ed25519 functions of crypto, which now import their packages on first use.
• paperspec.resources.path finds the schemas, SQL and vectors in an installed wheel and in an editable install alike.
• The programs in tools/ become subcommands of one command, paperspec, in src/paperspec/cli/: paperspec validate, load, check, build, stress and site, which the pixi tasks and the workflows run, and which python -m paperspec also runs. The scripts tools/*.py are gone, so python3 tools/check.py and the like no longer work; tools/ keeps the page that documents the commands.
• paperspec --help says which commands work anywhere and which need a checkout, each command's --help gives advice and examples, a misspelled command gets a suggestion, and paperspec --version prints the package version and the Unicode version it pins.
• paperspec load refuses --built-from and --store without --bundle, where it used to ignore them.
• src/ is Apache-2.0, as tools/ was, and the package metadata states Apache-2.0 AND CC0-1.0 for the code and the data it carries.
• The pixi workspace installs the package editable, and a dev environment builds the sdist and the wheel with pixi run -e dev dist.
• Pushing a version tag publishes the package to the Gitea package registry through .gitea/workflows/publish.yml (deploy/); a release tag versions the package, and CONTRIBUTING.md says how that relates to the format version.

Unreleased

Two changes asked for by the first producer that compares a converted reading with a second witness (papervert with a JATS file). Both are minor changes in the sense of spec/00: no valid file becomes invalid, and no hash, ID or key changes; the next release numbers them.

• Alignment has a substitution step (spec/05 §5.6, step 3): in each stretch between pairs, an unpaired sentence of the first reading is paired with the first unpaired sentence of the second whose similarity (2L / (|a| + |b|) over code points of the lossy projections, L their longest common subsequence) is at least 0.8, both of prose length 20 or more. A sentence one reading misread is now one number, equation or word suspect rather than a dropped and an added one, so a producer re-aligning the same two readings can get different suspects than before; the pairs and suspects of readings with no such sentence are unchanged, as the first alignment vector is. paperspec.align gains lcs_length (bit-parallel, Hyyrö 2004) and similarity, and alignment.jsonl a second case, substitutions: a misread number, a misread word, a sentence too short to be paired this way and two sentences too unlike to be one.
• A suspect may name the witness its second reading came from (witness, a witness ID; spec/05 §5.7, suspect.schema.json), and a package whose suspect names a witness it does not have breaks doc.suspect.witness. No broken package exercises the rule yet: adding one changes the count of broken packages that the 0.1.0 entry above states, so it waits for the release.

Four more, asked for by editors who correct and discuss papers rather than readings. They are minor changes too: new operations, enumeration values and optional properties, with no valid file made invalid, no value of an existing file computed differently, and no existing vector case or package changed.

• Annotation categories (spec/18 §18.4, annotation.schema.json, the category term of the JSON-LD context): a comment, caveat or question body may carry a category from the fixed list comment, context, critique, question, correction, feedback and resource, which a producer that supports categories SHOULD set on every such body. The store's annotations table gains a nullable category column with its CHECK (sql/store.sql). The example layer's comment, question and reply gain categories, which changes their record IDs, and it gains a caveat; no other fixture changes.
• emend, a correction of the authors' own typo, spelling, grammar, punctuation or wording, where the reading matches the page and the page is wrong (spec/11 §11.6). The emended text keeps what was printed in a new optional run property, printed (spec/03 §3.5, runs.schema.json), so a leaf as printed is recovered from the package alone (3.6, paperspec.hashing.printed and as_printed), and runs with printed are never joined. The strict and lossy projections are of the runs as printed, so an emend changes the leaf's text_hash, own_hash and node_hash but not its match_hash or align_key, and alignment, corroboration and suspects never report an emendation (spec/05 §5.1). The gate refuses an emend in math, across runs, or that changes a digit or edits beside one, and refuses a replace, text insert or split inside an emended run (correction.emend).
• relink makes a run a citation of other references, a text run a cite run, or a cite run text, without changing its text, and resolve sets a reference's resolution in references.jsonl, keeping the one before in from (spec/11 §11.7); a package whose last resolve of a reference is not its resolution breaks doc.correction.resolution. Like an emend, neither lowers a leaf's level, and a receipt keeps counting across all three (spec/08 §8.1–8.2, receipts.carried_hashes).
• The rules correction.emend, correction.relink, correction.resolve and doc.correction.resolution are registered, the corrections table's CHECK takes the three operations (sql/doc.sql), corrections.State carries the reference records, and paperspec stress walks the three operations with their invariants. corrections.jsonl gains thirteen cases and projections.jsonl and projection-groups.jsonl two each; no existing case changes.