paperspec/docs/examples.md

Examples

Things to run, each with the output it gives. Run them from the repository's root. With pixi, pixi run <task> sets up the environment first; every command also works with plain Python 3.10 or later and standard command-line tools.

Run the checks

pixi run check

pixi run check lints the Markdown and then runs python tools/check.py, which you can also run directly. Each section prints how many checks it ran, and the last line is All checks pass. pixi run quick, or --quick, skips the slowest part: rebuilding everything and comparing it byte for byte.

Recompute a hash by hand

A sentence's text_hash is SHA-256 of its text, after Unicode NFC and with whitespace collapsed. For a sentence of plain text, that is a one-liner:

printf '%s' 'The two parameter sets agree to within 2% for every bond.' | sha256sum

175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a  -

The same value is in the package:

grep -o '"id":"j06tfhn".*' fixtures/packages/hale2026harmonic/nodes.jsonl | grep -o '"text_hash":"[0-9a-f]*"'

"text_hash":"175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a"

Check a package's manifest

Every file of a package is listed in MANIFEST.json with its SHA-256, and the manifest's own hash pins the whole package.

cd fixtures/packages/okafor2025open
sha256sum nodes.jsonl
grep -o '"path":"nodes.jsonl","sha256":"[0-9a-f]*"' MANIFEST.json

0fccd6492121fb91ebd1d32487e23ca5b30ad92627e7d3a0801d77549e1a2f94  nodes.jsonl
"path":"nodes.jsonl","sha256":"0fccd6492121fb91ebd1d32487e23ca5b30ad92627e7d3a0801d77549e1a2f94"

Read a package with nothing but a JSON parser

Packages are plain JSON Lines. Save this script as outline.py; it prints a document's sections, figures, tables and equations.

import json, sys

pkg = sys.argv[1]
nodes = [json.loads(line) for line in open(pkg + "/nodes.jsonl", encoding="utf-8")]
by_id = {n["id"]: n for n in nodes}
children = {}
for n in nodes:
    children.setdefault(n["parent"], []).append(n)


def text(n):
    """The plain projection of a node with runs."""
    out = []
    for r in n["content"]:
        if r["kind"] == "math":
            out.append("$" + r["source"] + "$")
        elif r["kind"] == "equation":
            out.append("$$" + by_id[r["node"]]["content"]["source"] + "$$")
        else:
            out.append(r["text"])
    return "".join(out)


for n in nodes:
    if n["kind"] == "section":
        heading = children[n["id"]][0]
        print(n["id"], n.get("label", ""), " ".join(text(s) for s in children[heading["id"]]))
    elif n["kind"] in ("figure", "table", "equation"):
        print(n["id"], "  ", n["kind"], n.get("label", ""))

python3 outline.py fixtures/packages/hale2026harmonic

bzqvbhd 1 Introduction
qn08tj9    equation (1)
f0tj6ca 2 Methods
ajfvq4s    figure Figure 1
gm5z4rx    table Table 1

Follow an address

hale2026harmonic#f929eq9[29:44] names code points 29 to 44 of sentence f929eq9. With text() from the script above:

print(text(by_id["f929eq9"])[29:44])

first test case

Offsets count Unicode code points; in Python, string indexes already do. In JavaScript, convert to UTF-16 units first; vectors/offsets.jsonl gives both for text with characters outside the Basic Multilingual Plane.

Load packages into SQLite

pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic fixtures/packages/okafor2025open fixtures/packages/smithlab2027charges

Without pixi, run python3 tools/load.py with the same arguments.

hale2026harmonic: 75 nodes
okafor2025open: 15 nodes
smithlab2027charges: 15 nodes

The tables are those of sql/doc.sql and sql/store.sql, and the same statements run on Dolt and PostgreSQL. Here are four queries, in Python's sqlite3 so that nothing else is needed.

import json, sqlite3

db = sqlite3.connect("lab.db")

# Sentences the converter left below F2. Receipts can raise them; see below.
for row in db.execute("SELECT id, fidelity FROM nodes WHERE doc_key = 'hale2026harmonic' "
                      "AND kind = 'sentence' AND fidelity IN ('F0', 'F1') ORDER BY seq"):
    print(row)

# Every sentence that cites reference 2: the citation graph is sentence-level.
for row in db.execute("SELECT c.sentence FROM citations c JOIN refs r "
                      "ON r.doc_key = c.doc_key AND r.node = c.ref_node "
                      "WHERE c.doc_key = 'hale2026harmonic' AND r.label = '2'"):
    print(row)

# Who signed what.
for row in db.execute("SELECT r.signer, r.scope, COUNT(*) FROM receipts r "
                      "JOIN receipt_nodes rn ON rn.receipt = r.id GROUP BY r.id ORDER BY r.made_at"):
    print(row)

# Table 1's body, row by row.
rows = db.execute("SELECT r.id FROM nodes r JOIN nodes b ON b.doc_key = r.doc_key AND b.id = r.parent "
                  "WHERE r.doc_key = 'hale2026harmonic' AND r.kind = 'row' AND b.kind = 'table-body' "
                  "ORDER BY r.seq").fetchall()
for (row,) in rows:
    cells = db.execute("SELECT content FROM nodes WHERE doc_key = 'hale2026harmonic' AND parent = ? ORDER BY ord",
                       (row,)).fetchall()
    print(" | ".join("".join(r.get("text", "") for r in json.loads(c)) for (c,) in cells))

('e4fkseg', 'F1')
('r1gacps', 'F0')
('eajq93t', 'F0')
('f929eq9',)
('ana', 'targeted:lode', 3)
('ana', 'targeted:lode', 23)
('ana', 'section:bzqvbhd', 2)
('cara', 'page:2', 3)
('ana', 'targeted:captions', 3)
('ben', 'qa:eqgbbpwb6725v', 10)
('ben', 'qa:gena76nkb6cnc', 3)
Bond parameters
CT–CT | 310.0 | 1.526 | 305.2 | 1.5260
CT–HC | 340.0 | 1.090 | 338.7 | 1.0900
OW–HW | 553.0 | 0.9572 | 548.1 | 0.9570

The first query reads the level the converter recorded, not the final one. Receipts by ana raise e4fkseg and r1gacps to F3; eajq93t stays at F0 because the only receipt that covered it was revoked. Computing the final level means checking each receipt's signature, revocations and spot-checks, as chapter 8 describes.

Test your implementation against a vector

Every vector file has one case per line, with its inputs and expected outputs. A harness for the projections looks like this, with your functions in place of the comments:

import json

for line in open("vectors/projections.jsonl", encoding="utf-8"):
    case = json.loads(line)
    runs = case["runs"]
    # assert my_plain(runs) == case["plain"]
    # assert my_strict(runs) == case["strict"]
    # assert my_lossy(runs) == case["lossy"]
    # assert my_sha256(case["strict"]) == case["match_hash"]
    print(case["id"], case["strict"])

vectors/README.md lists every file and what it tests.

Break something on purpose

Change one digit in a fixture and run the checks:

sed -i 's/1718 cm/1719 cm/' fixtures/packages/okafor2025open/nodes.jsonl
pixi run quick

The checker reports the package's manifest hash no longer matches, and that unloading it from the database no longer gives the same bytes. Restore the file with git checkout fixtures/packages/okafor2025open/nodes.jsonl. The broken packages in fixtures/invalid/ are this kind of change, each with the rule it breaks.