alexmandClaude
88379d152c
The six console scripts (paperspec-check, -build, -validate, -load, -stress, -site) become subcommands of a single `paperspec` command: paperspec validate, load, check, build, stress and site. `python -m paperspec COMMAND` runs the same thing. - paperspec/cli/main.py builds the whole parser, apart from the commands, so `paperspec --help` and `paperspec COMMAND --help` work without a checkout or the tools extra; a command's module is imported only when it runs. - `paperspec` with no command prints the overview: what each command does, examples, and which work anywhere (validate, load) and which need a checkout (check, build, stress, site). Each command's help has a description with advice, and worked examples; a misspelled command gets a suggestion (argparse suggest_on_error); `--version` prints the version and the pinned Unicode version. - Each command module exposes run(args) and returns the exit status. Arguments that parse but do not fit together raise UsageError, which prints the command's usage and exits 2: load now also refuses --built-from or --store without --bundle. - The failure messages say what to do: install paperspec[tools], clone the repository, rerun stress with the printed seed, or run paperspec build when the rebuild check differs. check and stress say how many failures they did not list. - check runs the rebuild as `python -m paperspec build`. - pyproject installs one script, `paperspec`; the pixi tasks and the docs use the subcommands. pixi run check passes (run here as `python -m paperspec check` with src/ on the path); the rebuild is byte-identical. pixi.lock still needs `pixi lock`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Test vectors
Inputs and the exact outputs every implementation must compute.
This folder is for three readers. New to the problem? The first section says what a test vector is and why one exists. Writing a reader, writer, converter or checker? Read the procedure for running a file, then use the table to find the vectors and chapters for the rule you are implementing. Judging the specification? The last section says what the vectors establish and what they leave open.
What a test vector is
Two programs can read a rule written in prose differently, and then the values they compute no longer match. These formats store hashes of text, so a mismatch is visible and costly. A hash is a short fixed-length fingerprint computed from data: the same data always gives the same hash, and in practice any change gives a different one. These formats use SHA-256, written as 64 lowercase hexadecimal digits (1.3). The same sentence gets two hashes when one program collapses runs of spaces before hashing and another does not, and the second program sees a change that never happened.
A test vector removes the doubt. It is one case: an input and the exact output that the specification requires for it. An implementation passes a case when its own output for that input is identical to the expected one. The vectors are normative, like the chapters: where a chapter and a vector disagree, the disagreement is a bug (spec/README.md).
One case from normalize.jsonl, which pins the text normalization of 1.4:
{"id":"collapse","input":"a \t\n b","output":"a b","sha256":"c8687a08aa5d6ed2044328fa6a697ab8e96dc34291e8c2034ae8c38e6fcc6d65"}
The input is the letter a, a space, a tab, a newline, a space and the letter b.
The rule collapses every run of whitespace to one space, so the output is a b, and sha256 is the SHA-256 of that output.
JSON is a text format for structured data, and a file of one JSON value per line is called JSON Lines.
Each file holds one case per line, in canonical JSON, with an id, its inputs, and its expected outputs.
Canonical JSON is the one agreed way of writing a JSON value, the JSON Canonicalization Scheme of RFC 8785, a numbered Internet standard (RFC means Request for Comments), plus the repository's own rules (1.1): no whitespace between tokens, object members in a fixed order, and numbers written one way.
The same data is always the same bytes when every file is written this way, so it hashes the same.
A format's identifier, for example scienting-id/1, appears inside several cases.
The scienting- prefix is the formats' namespace; it has no connection to the name of this repository.
How to run a vector file
Run the vectors in your own test suite, against your own implementation.
docs/examples.md shows the shape of a harness for projections.jsonl.
The procedure is the same for every file:
- Read the file as UTF-8 (the standard Unicode text encoding) and split it at U+000A only.
normalize.jsonlcontains a raw U+2028 (line separator) inside one case, so a reader that splits at every Unicode line break, such as Python'sstr.splitlines(), cuts that case in two. Python'sfor line in open(path, encoding="utf-8")splits correctly. - Parse each line as JSON.
- Find the inputs and the expected outputs of the case.
The table below says which fields are which, and the chapter it links defines the rule.
Only six files,
alignment,corrections,grading,resolve,rightsandtargets, put the expected outputs underexpect. In the other nineteen the expected outputs are fields beside the inputs, such asoutput,sha256,canonicalorparsed. - Compute each output from the inputs with your implementation.
- Compare each computed output with the expected one. Strings must be identical. Objects and arrays are compared as values, which is how the reference checker compares them.
- Report every difference with the case's
id.
Some files change the procedure:
canonical-json.jsonlgives JSON text ininput_json. Parse its numbers as IEEE 754 doubles (the 64-bit floating-point numbers most languages use), as a JavaScript engine does. Thenumberscase contains123456789012345678901, which a parser that keeps exact integers reads differently; the expected output is123456789012345680000.projection-groups.jsonlhas no inputs of its own. Each of its cases lists ids fromprojections.jsonland names a field in which those cases must be equal (same) or unequal (differ). Compute the field for each listed case and compare across the group.fingerprints.jsonlneeds the download-stamp rules in registry/stamps/scienting-stamps-1.json, which each case names inrules.resolve.jsonl,grading.jsonlandtargets.jsonlname a package by a path from the repository root. All three usefixtures/packages/hale2026harmonic.
Several files also mix more than one kind of case, told apart by their fields.
ids.jsonl has minting cases and letter-first cases, keyed.jsonl and signatures.jsonl have a case for each derivation or record, spotcheck.jsonl has sample-size cases and one full sample, and rights.jsonl has stage checks (a stage field), backend checks (a backend field) and one combining case (neither).
The reference checker in tools/ runs all twenty-five files, together with the rest of the repository's checks:
paperspec check --quick
It needs Python 3.14 and two packages, jsonschema and cryptography, and has no option to run one file.
It tests the reference implementation in src/paperspec/ against the vectors.
It does not test yours.
What each file tests
The table below uses a few terms that the chapters define:
- A code point is one Unicode character; UTF-16 is an encoding that writes some characters as two units, so a JavaScript string offset can differ from a code-point offset (1.5).
- An HMAC is a hash computed under a secret key, so that someone without the key cannot recompute it (5.8).
- A Merkle root is one hash that commits to a list of items by hashing them in a binary tree (8.5).
- A page fingerprint is a hash of a page's text and images with download stamps removed, so two differently stamped downloads of one paper share it (7.1).
- The strict and lossy projections are two canonical forms of a sentence's text, one to certify that two readings agree and one to propose that they might (5.3).
docs/glossary.md defines the rest.
| File | Tests | Chapter |
|---|---|---|
canonical-json.jsonl |
input_json, parsed with every number as an IEEE 754 double, and its canonical output. |
1.1 |
normalize.jsonl |
Normalization for hashing, and the SHA-256 of the result. | 1.4 |
ids.jsonl |
Minting payloads and node IDs, and letter-first encodings of digests. | 4 |
notation.jsonl |
Each math source's canonical linear form, or null when it is complex. | 5.4 |
projections.jsonl |
One-sentence runs: plain, normalized, strict and lossy projections, complex keys and the three text values. | 5.3 |
projection-groups.jsonl |
Which cases must share, or must differ in, a value: H₂O in six spellings, cm⁻¹, 10⁻³, K_r, ligatures, the micro sign, "103" against 10³. | 5.3 |
offsets.jsonl |
Code-point spans and their UTF-16 equivalents. | 1.5 |
hashes.jsonl |
Every node of a small document, its own content and its five values. | 5.1 |
addresses.jsonl |
How addresses parse; null for invalid ones. | 6.1 |
keyed.jsonl |
The team key and its ID, an HMAC, page hashes, a page-derived key, and what someone without the paper cannot check. | 5.8 |
fingerprints.jsonl |
Pages as text lines, the stamp rules that fired, and the fingerprint; two downloads that must agree, and a change that must not. | 7.1 |
merkle.jsonl |
Receipt items and their Merkle roots. | 8.5 |
signatures.jsonl |
The RFC 8032 test vector; a receipt's record hash, signing payload and signature; did:key identifiers for an Ed25519 and an ML-DSA-65 key; and ML-DSA-65 (FIPS 204): NIST's key-generation case and a deterministic signature. |
8.4 |
spotcheck.jsonl |
Seed payloads, seeds, sample sizes and samples. | 8.8 |
bounds.jsonl |
Reviewer error bounds as reported. | 8.9 |
alignment.jsonl |
Two readings of a document, their pairs and their suspects. | 5.6 |
corrections.jsonl |
Each operation and refusal on a small document: the state before, the correction, and the record and state after, or the rule broken. | 11 |
resolve.jsonl |
Anchors made on the example paper's first reading, and how each resolves after its corrections. | 6.5 |
rights.jsonl |
Stage checks with policies and expiry, the combining rule, and backend conditions. | 14 |
grading.jsonl |
Exercises, submissions and the grades they earn. | 19.4 |
targets.jsonl |
The target set of targeted F3 for the example paper under each digit rule, given the anchored nodes. | 8.10 |
keys.jsonl |
Work keys: Scholar parts, identities, hash suffixes and document keys, with accented names, missing years and a collision. | 2.4 |
proofs.jsonl |
RFC 9162 inclusion paths for every item of small trees, consistency proofs for every prefix, the empty root, and the example paper's history. | 8.5, 11.5 |
vrf.jsonl |
The VRF of spot-check seeds: RFC 9381's Example 16, two more inputs for its key, the example paper's seed, and proofs that must not verify (beta null). Bytes are hex. |
8.8 |
checkpoint.jsonl |
A transparency-log checkpoint: the entries, their root, the signed note and its signers. | 23 |
Resolve those paths against the files in that package only.
What the vectors prove, and what they do not
The specification, and the vectors with it, are at version 0.1.0 (0.3).
The specification has 32 formats: 19 are candidate and 13 are provisional in registry/formats.json.
A candidate format changes only with a recorded reason and a new or changed vector (22.5).
A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in src/paperspec/.
That code computed the expected outputs, and pixi run build regenerates them (tools/).
The vectors hold 311 cases in all, some of them small: hashes.jsonl has one document, alignment.jsonl one pair of readings, offsets.jsonl two cases.
The vectors do not show the following.
- Conformance: the vectors are one part of the suite in chapter 22.
A conforming implementation must also accept every valid package in
fixtures/packages/, and refuse every package infixtures/invalid/, naming the rule it breaks if it can. No vector tests package validation. - Independent agreement: the only program in this repository that runs the vectors is the reference checker, and it uses the same oracle that produced them. A rule that the oracle and the chapter both misstate would pass. A separately written implementation that passes is evidence the repository cannot supply itself, and a disagreement between a chapter and a vector is a bug to report.
- Rules the vectors skip:
canonical-json.jsonlhas five cases, none for the repository's rule that a lone surrogate,NaNor an infinity has no canonical form.merkle.jsonlhas roots for 1, 2, 3, 4, 5 and 7 items. A receipt with no items is invalid (8.5), so no case has an empty list, although the RFCs it follows define a hash for an empty list (RFC 6962, RFC 9162).- Thirteen formats are
provisional, and onlygrading.jsonlcarries any of their identifiers (exercise, submission and grade). The chapters of the others define values that no vector pins, such as a dataset's content hash (15), the hash that confirms a digitization (16) and a checklist's content hash (8.11). They are expected to change as implementations gain experience (0.4).
- Reading a PDF: how text lines are extracted from a PDF is outside the specification;
fingerprints.jsonlsupplies the lines, so it tests the fingerprint function only (7.1). - Faithful conversion: the vectors check what is computed from text, not whether a converter read the paper correctly. The formats record such an error and make it checkable through corrections and receipts (chapter 8, chapter 11); the vectors do not detect it.
- Security:
signatures.jsonlcontains the RFC 8032 test vector and one receipt, so it checks that your Ed25519 and record hashing give the stated bytes. The secrets inkeyed.jsonlandsignatures.jsonlare test values published in this repository; do not use them for anything real.
A vector that rests on a building block defined elsewhere pins this specification's use of it.
Canonical JSON is RFC 8785 with added rules.
Receipts use a Merkle tree that follows RFC 6962 (8.5); RFC 9162 replaces RFC 6962 and defines the same tree.
The node_hash of a node is a different construction: a hash of the canonical JSON of {own, children}.
Here own is the node's own hash and children holds the node_hash of each child (5.1), and hashes.jsonl pins it.
The comparison page covers how these formats differ from JATS (Journal Article Tag Suite), TEI (Text Encoding Initiative), W3C (World Wide Web Consortium) Web Annotation and the other neighbors.