paperspec/docs/examples.md

Examples

Things to run, each with the output it gives. Three readers can use this page. Each section opens with what it shows, in plain terms, for readers new to the problem, and every example uses the fixtures in this repository. The last sections point implementers (anyone writing a reader, writer, converter or checker) to the test vectors, which pin the exact behavior (the vector files are listed in vectors/README.md). This page makes no comparison with the neighboring standards. The formats are version 0.1.0, and each is candidate or provisional in registry/formats.json.

Run every command from the root of the repository, https://git.scient.ing/infra/paperspec. With pixi, pixi run <task> sets up the environment first. Every command also works without pixi once the checkout is installed with pip install -e ".[tools]" on Python 3.14, given standard command-line tools (the sed command in the last section is the GNU version of sed).

Run the checks

This shows how to confirm that the repository's specification, fixtures and test vectors agree with each other.

pixi run check

pixi run check lints the Markdown, then runs paperspec check, the repository's reference checker. Each section prints how many checks it ran, and the last line is All checks pass. pixi run quick, or paperspec check --quick, skips the slowest part, which rebuilds everything and compares it byte for byte. Without pixi, the Markdown lint is a separate step (markdownlint-cli2); paperspec check does not run it.

Recompute a hash by hand

This shows that a hash in a package is not magic: you can reproduce it with a standard tool. A hash is a fixed-length fingerprint of some text; change one character and the fingerprint changes. A sentence's text_hash is the SHA-256 of its text after two steps. The text is converted to Unicode NFC (a standard form that writes each accented or composed character one way), and each run of whitespace is collapsed to one space, as chapter 1 defines. For a sentence with math or citations, the text is its plain projection (chapter 5). For a sentence of plain text, the whole computation is a one-liner:

printf '%s' 'The two parameter sets agree to within 2% for every bond.' | sha256sum

175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a  -

The same value is in the package, in the nodes.jsonl of hale2026harmonic-damb:

grep -o '"id":"v3063qv".*' fixtures/packages/hale2026harmonic-damb/nodes.jsonl | grep -o '"text_hash":"[0-9a-f]*"'

"text_hash":"175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a"

Check a package's manifest

This shows how a reader detects that a package's files have changed. A package is a folder of JSON (JavaScript Object Notation, a plain-text format for structured data) files describing one paper. MANIFEST.json lists every other file of a package with its SHA-256 and size, and the SHA-256 of the manifest itself pins the whole package (chapter 12).

cd fixtures/packages/okafor2025solvent-acth
sha256sum nodes.jsonl
grep -o '"path":"nodes.jsonl","sha256":"[0-9a-f]*"' MANIFEST.json

09c9b99eed97987e51dd0f85461ad9ad8768a0769ba7facfb7e0879d742660a3  nodes.jsonl
"path":"nodes.jsonl","sha256":"09c9b99eed97987e51dd0f85461ad9ad8768a0769ba7facfb7e0879d742660a3"

Read a package with nothing but a JSON parser

This shows that a package needs no special software to read. JSON Lines is a text format with one JSON object per line, here one node (a section, sentence, figure, table or equation) per line of nodes.jsonl. Save this script as outline.py. It prints a document's sections, figures, tables and equations.

import json, sys

pkg = sys.argv[1]
nodes = [json.loads(line) for line in open(pkg + "/nodes.jsonl", encoding="utf-8")]
by_id = {n["id"]: n for n in nodes}
children = {}
for n in nodes:
    children.setdefault(n["parent"], []).append(n)


def text(n):
    """The plain projection of a node with runs."""
    out = []
    for r in n["content"]:
        if r["kind"] == "math":
            out.append("$" + r["source"] + "$")
        elif r["kind"] == "equation":
            out.append("$$" + by_id[r["node"]]["content"]["source"] + "$$")
        else:
            out.append(r["text"])
    return "".join(out)


for n in nodes:
    if n["kind"] == "section":
        heading = children[n["id"]][0]
        print(n["id"], n.get("label", ""), " ".join(text(s) for s in children[heading["id"]]))
    elif n["kind"] in ("figure", "table", "equation"):
        print(n["id"], "  ", n["kind"], n.get("label", ""))

python3 outline.py fixtures/packages/hale2026harmonic-damb

p36k8t6 1 Introduction
tkxsmts    equation (1)
d400rz3 2 Methods
v7cx6qs    figure Figure 1
hj4s7pb    table Table 1

Follow an address

This shows how a citation points inside a sentence. An address names a node, and optionally a span of its text (chapter 6). hale2026harmonic-damb#hca6dc5[29:44] names code points 29 to 44 of sentence hca6dc5. With text() from the script above:

print(text(by_id["hca6dc5"])[29:44])

first test case

Offsets count Unicode code points, one per character; in Python, string indexes already do. JavaScript strings index UTF-16 units, so convert first. For text with characters outside the Basic Multilingual Plane the two counts differ, and vectors/offsets.jsonl gives both.

Load packages into SQLite

This shows that the same packages can live in a database and be queried with SQL.

pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic-damb fixtures/packages/okafor2025solvent-acth fixtures/packages/writer2027fixed-ynr7.v3

Without pixi, run paperspec load with the same arguments.

hale2026harmonic-damb: 75 nodes
okafor2025solvent-acth: 15 nodes
writer2027fixed-ynr7.v3: 15 nodes

The tables are those of sql/doc.sql and sql/store.sql, which run unchanged on Dolt and PostgreSQL as well (chapter 13); paperspec check exercises them on SQLite only, and this repository has not yet run them on Dolt or PostgreSQL. Here are four queries, in Python's sqlite3 so that nothing else is needed.

import json, sqlite3

db = sqlite3.connect("lab.db")

# Sentences the converter left below F2. Receipts can raise them; see below.
for row in db.execute("SELECT id, fidelity FROM nodes WHERE doc_key = 'hale2026harmonic-damb' "
                      "AND kind = 'sentence' AND fidelity IN ('F0', 'F1') ORDER BY seq"):
    print(row)

# Every sentence that cites reference 2: the citation graph is sentence-level.
for row in db.execute("SELECT c.sentence FROM citations c JOIN refs r "
                      "ON r.doc_key = c.doc_key AND r.node = c.ref_node "
                      "WHERE c.doc_key = 'hale2026harmonic-damb' AND r.label = '2'"):
    print(row)

# Who signed what.
for row in db.execute("SELECT r.signer, r.scope, COUNT(*) FROM receipts r "
                      "JOIN receipt_nodes rn ON rn.receipt = r.id GROUP BY r.id ORDER BY r.made_at"):
    print(row)

# Table 1's body, row by row.
rows = db.execute("SELECT r.id FROM nodes r JOIN nodes b ON b.doc_key = r.doc_key AND b.id = r.parent "
                  "WHERE r.doc_key = 'hale2026harmonic-damb' AND r.kind = 'row' AND b.kind = 'table-body' "
                  "ORDER BY r.seq").fetchall()
for (row,) in rows:
    cells = db.execute("SELECT content FROM nodes WHERE doc_key = 'hale2026harmonic-damb' AND parent = ? ORDER BY ord",
                       (row,)).fetchall()
    print(" | ".join("".join(r.get("text", "") for r in json.loads(c)) for (c,) in cells))

('ec5jc93', 'F1')
('a4jrz2t', 'F0')
('xjns57z', 'F0')
('hca6dc5',)
('ana', 'targeted:paperclaim', 3)
('ana', 'targeted:paperclaim', 23)
('ana', 'section:p36k8t6', 2)
('cara', 'page:2', 3)
('ana', 'targeted:captions', 3)
('ben', 'qa:fhhvg9qrzqp0n', 10)
('ben', 'qa:gx2s84z9jf9xv', 3)
Bond parameters
CT–CT | 310.0 | 1.526 | 305.2 | 1.5260
CT–HC | 340.0 | 1.090 | 338.7 | 1.0900
OW–HW | 553.0 | 0.9572 | 548.1 | 0.9570

The first query reads the level the converter recorded, not the final one. A receipt is a signed record that a reviewer checked certain nodes against the page. Receipts by ana raise ec5jc93 and a4jrz2t to F3; xjns57z stays at F0 because the only receipt that covered it, by cara, was revoked. To compute the final level, check each receipt's signature, revocations and spot-checks, as chapter 8 describes. The scope column of the third query shows the value targeted:paperclaim, which is data in the fixture, copied here exactly as the query prints it.

Follow a traced citation

This shows how a citation leads to the passage it rests on, and how to tell that it still does. A trace is an annotation from the sentence that cites a paper to a passage of that paper, with the reference it follows (chapter 18). The draft in the fixtures cites the main example once, and its layer of traces holds three: a model's, a person's, and a candidate a person rejected. The Python package paperspec.traces judges each against the current state of both papers:

import json
from paperspec import package, traces

draft = package.read_state("fixtures/packages/writer2027fixed-ynr7.v3")
paper = package.read_state("fixtures/packages/hale2026harmonic-damb")
same_as = json.load(open("fixtures/packages/hale2026harmonic-damb/document.json"))["work"]["same_as"]
annotations = [json.loads(line) for line in open("fixtures/records/traces/annotations.jsonl", encoding="utf-8")]
for a in annotations:
    if a["body"]["kind"] == "trace":
        r = traces.resolve(a, draft, paper, same_as)
        print(a["status"], r["result"], r["open"], traces.disputed(a["id"], annotations))

confirmed current smithlab/hale2026harmonic-damb#mhp0at8@ee42c4[48:60] True
rejected current smithlab/hale2026harmonic-damb#kea5026@b836d5[81:105] False
confirmed current smithlab/hale2026harmonic-damb#hj4s7pb@6c6fba False

Each line gives the trace's status, its outcome, the current address of the passage it leads to, and whether a reply disputes it. The first trace is disputed by a reply that says the table is the better passage, which is where the third trace leads. The outcomes here are all current; vectors/traces.jsonl corrects the draft or the paper and shows the others: changed when the passage was reworded but is still found, stale when it is gone, unlinked when the sentence no longer cites the reference, and misresolved when the reference now resolves to another paper. Resolving a trace also lists the citations inside the passage, so a chain of "as previously reported" is followed one trace at a time.

Test your implementation against a vector

A test vector is a recorded input together with the output every correct implementation must produce. Every vector file has one case per line, with its inputs and expected outputs. A harness for the projections looks like this, with your functions in place of the comments:

import json

for line in open("vectors/projections.jsonl", encoding="utf-8"):
    case = json.loads(line)
    runs = case["runs"]
    # assert my_plain(runs) == case["plain"]
    # assert my_strict(runs) == case["strict"]
    # assert my_lossy(runs) == case["lossy"]
    # assert my_sha256(case["strict"]) == case["match_hash"]
    print(case["id"], case["strict"])

vectors/README.md lists every file, what it tests and the chapter that defines it.

Break something on purpose

This shows what the checks catch. Change one digit in a fixture and run the checks:

sed -i 's/1718 cm/1719 cm/' fixtures/packages/okafor2025solvent-acth/nodes.jsonl
pixi run quick

The checker reports three failures: the SHA-256 of nodes.jsonl no longer matches the manifest, unloading the package from the database no longer gives the same bytes, and the package hash differs after unloading. Restore the file with git checkout fixtures/packages/okafor2025solvent-acth/nodes.jsonl. The broken packages in fixtures/invalid/ are this kind of change, each with the rule it breaks. git checkout restores a tracked file, so run this only in a clone you can reset.