paperspec/vectors/README.md

Test vectors

Inputs and the exact outputs every implementation must compute.

This folder is for three readers. New to the problem? The first section says what a test vector is and why one exists. Writing a reader, writer, converter or checker? Read the procedure for running a file, then use the table to find the vectors and chapters for the rule you are implementing. Judging the specification? The last section says what the vectors establish and what they leave open.

What a test vector is

Two programs can read a rule written in prose differently, and then the values they compute no longer match. These formats store hashes of text, so a mismatch is visible and costly. A hash is a short fixed-length fingerprint computed from data: the same data always gives the same hash, and in practice any change gives a different one. These formats use SHA-256, written as 64 lowercase hexadecimal digits (1.3). The same sentence gets two hashes when one program collapses runs of spaces before hashing and another does not, and the second program sees a change that never happened.

A test vector removes the doubt. It is one case: an input and the exact output that the specification requires for it. An implementation passes a case when its own output for that input is identical to the expected one. The vectors are normative, like the chapters: where a chapter and a vector disagree, the disagreement is a bug (spec/README.md).

One case from normalize.jsonl, which pins the text normalization of 1.4:

{"id":"collapse","input":"a \t\n b","output":"a b","sha256":"c8687a08aa5d6ed2044328fa6a697ab8e96dc34291e8c2034ae8c38e6fcc6d65"}

The input is the letter a, a space, a tab, a newline, a space and the letter b. The rule collapses every run of whitespace to one space, so the output is a b, and sha256 is the SHA-256 of that output.

JSON is a text format for structured data, and a file of one JSON value per line is called JSON Lines. Each file holds one case per line, in canonical JSON, with an id, its inputs, and its expected outputs. Canonical JSON is the one agreed way of writing a JSON value, the JSON Canonicalization Scheme of RFC 8785, a numbered Internet standard (RFC means Request for Comments), plus the repository's own rules (1.1): no whitespace between tokens, object members in a fixed order, and numbers written one way. The same data is always the same bytes when every file is written this way, so it hashes the same.

A format's identifier, for example scienting-id/1, appears inside several cases. The scienting- prefix is the formats' namespace; it has no connection to the name of this repository.

How to run a vector file

Run the vectors in your own test suite, against your own implementation. docs/examples.md shows the shape of a harness for projections.jsonl. The procedure is the same for every file:

1. Read the file as UTF-8 (the standard Unicode text encoding) and split it at U+000A only. normalize.jsonl contains a raw U+2028 (line separator) inside one case, so a reader that splits at every Unicode line break, such as Python's str.splitlines(), cuts that case in two. Python's for line in open(path, encoding="utf-8") splits correctly.
2. Parse each line as JSON.
3. Find the inputs and the expected outputs of the case. The table below says which fields are which, and the chapter it links defines the rule. Only five files, alignment, corrections, grading, resolve and rights, put the expected outputs under expect. In the other fifteen the expected outputs are fields beside the inputs, such as output, sha256, canonical or parsed.
4. Compute each output from the inputs with your implementation.
5. Compare each computed output with the expected one. Strings must be identical. Objects and arrays are compared as values, which is how the reference checker compares them.
6. Report every difference with the case's id.

Some files change the procedure:

• canonical-json.jsonl gives JSON text in input_json. Parse its numbers as IEEE 754 doubles (the 64-bit floating-point numbers most languages use), as a JavaScript engine does. The numbers case contains 123456789012345678901, which a parser that keeps exact integers reads differently; the expected output is 123456789012345680000.
• projection-groups.jsonl has no inputs of its own. Each of its cases lists ids from projections.jsonl and names a field in which those cases must be equal (same) or unequal (differ). Compute the field for each listed case and compare across the group.
• fingerprints.jsonl needs the download-stamp rules in registry/stamps/scienting-stamps-1.json, which each case names in rules.
• resolve.jsonl and grading.jsonl name a package by a path from the repository root. Both files use fixtures/packages/hale2026harmonic.

Several files also mix more than one kind of case, told apart by their fields. ids.jsonl has minting cases and letter-first cases, keyed.jsonl and signatures.jsonl have a case for each derivation or record, spotcheck.jsonl has sample-size cases and one full sample, and rights.jsonl has stage checks (a stage field), backend checks (a backend field) and one combining case (neither).

The reference checker in tools/ runs all twenty files, together with the rest of the repository's checks:

python3 tools/check.py --quick

It needs Python 3.10 or later and has no option to run one file. It tests the reference implementation in tools/oracle/ against the vectors. It does not test yours.

What each file tests

The table below uses a few terms that the chapters define:

• A code point is one Unicode character; UTF-16 is an encoding that writes some characters as two units, so a JavaScript string offset can differ from a code-point offset (1.5).
• An HMAC is a hash computed under a secret key, so that someone without the key cannot recompute it (5.8).
• A Merkle root is one hash that commits to a list of items by hashing them in a binary tree (8.5).
• A page fingerprint is a hash of a page's text and images with download stamps removed, so two differently stamped downloads of one paper share it (7.1).
• The strict and lossy projections are two canonical forms of a sentence's text, one to certify that two readings agree and one to propose that they might (5.3).

docs/glossary.md defines the rest.

File	Tests	Chapter
canonical-json.jsonl	input_json, parsed with every number as an IEEE 754 double, and its canonical output.	1.1
normalize.jsonl	Normalization for hashing, and the SHA-256 of the result.	1.4
ids.jsonl	Minting payloads and node IDs, and letter-first encodings of digests.	4
notation.jsonl	Each math source's canonical linear form, or null when it is complex.	5.4
projections.jsonl	One-sentence runs: plain, normalized, strict and lossy projections, complex keys and the three text values.	5.3
projection-groups.jsonl	Which cases must share, or must differ in, a value: H₂O in six spellings, cm⁻¹, 10⁻³, K_r, ligatures, the micro sign, "103" against 10³.	5.3
offsets.jsonl	Code-point spans and their UTF-16 equivalents.	1.5
hashes.jsonl	Every node of a small document, its own content and its five values.	5.1
addresses.jsonl	How addresses parse; null for invalid ones.	6.1
keyed.jsonl	The team key and its ID, an HMAC, page hashes, a page-derived key, and what someone without the paper cannot check.	5.8
fingerprints.jsonl	Pages as text lines, the stamp rules that fired, and the fingerprint; two downloads that must agree, and a change that must not.	7.1
merkle.jsonl	Receipt items and their Merkle roots.	8.5
signatures.jsonl	The RFC 8032 test vector, and a receipt's record hash, signing payload and signature.	8.4
spotcheck.jsonl	Seed payloads, seeds, sample sizes and samples.	8.8
bounds.jsonl	Reviewer error bounds as reported.	8.9
alignment.jsonl	Two readings of a document, their pairs and their suspects.	5.6
corrections.jsonl	Each operation and refusal on a small document: the state before, the correction, and the record and state after, or the rule broken.	11
resolve.jsonl	Anchors made on the example paper's first reading, and how each resolves after its corrections.	6.5
rights.jsonl	Stage checks with policies and expiry, the combining rule, and backend conditions.	14
grading.jsonl	Exercises, submissions and the grades they earn.	19.4

Resolve those paths against the files in that package only.

What the vectors prove, and what they do not

The specification, and the vectors with it, are at version 0.1.0 (0.3). The specification has 27 formats: 20 are candidate and seven are provisional in registry/formats.json. A candidate format changes only with a recorded reason and a new or changed vector (22.5).

A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in tools/oracle/. That code computed the expected outputs, and pixi run build regenerates them (tools/). The vectors hold 203 cases in all, some of them small: hashes.jsonl has one document, alignment.jsonl one pair of readings, offsets.jsonl and signatures.jsonl two cases each.

The vectors do not show the following.

• Conformance: the vectors are one part of the suite in chapter 22. A conforming implementation must also accept every valid package in fixtures/packages/, and refuse every package in fixtures/invalid/, naming the rule it breaks if it can. No vector tests package validation.
• Independent agreement: the only program in this repository that runs the vectors is the reference checker, and it uses the same oracle that produced them. A rule that the oracle and the chapter both misstate would pass. A separately written implementation that passes is evidence the repository cannot supply itself, and a disagreement between a chapter and a vector is a bug to report.
• Rules the vectors skip:
  • canonical-json.jsonl has five cases, none for the repository's rule that a lone surrogate, NaN or an infinity has no canonical form.
  • merkle.jsonl has roots for 1, 2, 3, 4, 5 and 7 items. A receipt with no items is invalid (8.5), so no case has an empty list, although the RFCs it follows define a hash for an empty list (RFC 6962, RFC 9162).
  • Seven formats are provisional, and no case carries their identifiers. Their chapters define values that no vector pins, such as a dataset's content hash (15) and the hash that confirms a digitization (16). They are expected to change as implementations gain experience (0.4).
• Reading a PDF: how text lines are extracted from a PDF is outside the specification; fingerprints.jsonl supplies the lines, so it tests the fingerprint function only (7.1).
• Faithful conversion: the vectors check what is computed from text, not whether a converter read the paper correctly. The formats record such an error and make it checkable through corrections and receipts (chapter 8, chapter 11); the vectors do not detect it.
• Security: signatures.jsonl contains the RFC 8032 test vector and one receipt, so it checks that your Ed25519 and record hashing give the stated bytes. The secrets in keyed.jsonl and signatures.jsonl are test values published in this repository; do not use them for anything real.

A vector that rests on a building block defined elsewhere pins this specification's use of it. Canonical JSON is RFC 8785 with added rules. Receipts use a Merkle tree that follows RFC 6962 (8.5); RFC 9162 replaces RFC 6962 and defines the same tree. The node_hash of a node is a different construction: a hash of the canonical JSON of {own, children}. Here own is the node's own hash and children holds the node_hash of each child (5.1), and hashes.jsonl pins it. The comparison page covers how these formats differ from JATS (Journal Article Tag Suite), TEI (Text Encoding Initiative), W3C (World Wide Web Consortium) Web Annotation and the other neighbors [CHECK: docs/comparison.md does not exist in the reference repository; link it once it does].