# Examples Things to run, each with the output it gives. Run them from the repository's root. With [pixi](https://pixi.sh), `pixi run ` sets up the environment first; every command also works with plain Python 3.10 or later and standard command-line tools. ## Run the checks ```sh pixi run check ``` `pixi run check` lints the Markdown and then runs `python tools/check.py`, which you can also run directly. Each section prints how many checks it ran, and the last line is `All checks pass.` `pixi run quick`, or `--quick`, skips the slowest part: rebuilding everything and comparing it byte for byte. ## Recompute a hash by hand A sentence's `text_hash` is SHA-256 of its text, after Unicode NFC and with whitespace collapsed. For a sentence of plain text, that is a one-liner: ```sh printf '%s' 'The two parameter sets agree to within 2% for every bond.' | sha256sum ``` ```text 175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a - ``` The same value is in the package: ```sh grep -o '"id":"j06tfhn".*' fixtures/packages/hale2026harmonic/nodes.jsonl | grep -o '"text_hash":"[0-9a-f]*"' ``` ```text "text_hash":"175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a" ``` ## Check a package's manifest Every file of a package is listed in `MANIFEST.json` with its SHA-256, and the manifest's own hash pins the whole package. ```sh cd fixtures/packages/okafor2025open sha256sum nodes.jsonl grep -o '"path":"nodes.jsonl","sha256":"[0-9a-f]*"' MANIFEST.json ``` ```text 0fccd6492121fb91ebd1d32487e23ca5b30ad92627e7d3a0801d77549e1a2f94 nodes.jsonl "path":"nodes.jsonl","sha256":"0fccd6492121fb91ebd1d32487e23ca5b30ad92627e7d3a0801d77549e1a2f94" ``` ## Read a package with nothing but a JSON parser Packages are plain JSON Lines. Save this script as `outline.py`; it prints a document's sections, figures, tables and equations. ```python import json, sys pkg = sys.argv[1] nodes = [json.loads(line) for line in open(pkg + "/nodes.jsonl", encoding="utf-8")] by_id = {n["id"]: n for n in nodes} children = {} for n in nodes: children.setdefault(n["parent"], []).append(n) def text(n): """The plain projection of a node with runs.""" out = [] for r in n["content"]: if r["kind"] == "math": out.append("$" + r["source"] + "$") elif r["kind"] == "equation": out.append("$$" + by_id[r["node"]]["content"]["source"] + "$$") else: out.append(r["text"]) return "".join(out) for n in nodes: if n["kind"] == "section": heading = children[n["id"]][0] print(n["id"], n.get("label", ""), " ".join(text(s) for s in children[heading["id"]])) elif n["kind"] in ("figure", "table", "equation"): print(n["id"], " ", n["kind"], n.get("label", "")) ``` ```sh python3 outline.py fixtures/packages/hale2026harmonic ``` ```text bzqvbhd 1 Introduction qn08tj9 equation (1) f0tj6ca 2 Methods ajfvq4s figure Figure 1 gm5z4rx table Table 1 ``` ## Follow an address `hale2026harmonic#f929eq9[29:44]` names code points 29 to 44 of sentence `f929eq9`. With `text()` from the script above: ```python print(text(by_id["f929eq9"])[29:44]) ``` ```text first test case ``` Offsets count Unicode code points; in Python, string indexes already do. In JavaScript, convert to UTF-16 units first; [vectors/offsets.jsonl](../vectors/offsets.jsonl) gives both for text with characters outside the Basic Multilingual Plane. ## Load packages into SQLite ```sh pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic fixtures/packages/okafor2025open fixtures/packages/smithlab2027charges ``` Without pixi, run `python3 tools/load.py` with the same arguments. ```text hale2026harmonic: 75 nodes okafor2025open: 15 nodes smithlab2027charges: 15 nodes ``` The tables are those of [sql/doc.sql](../sql/doc.sql) and [sql/store.sql](../sql/store.sql), and the same statements run on Dolt and PostgreSQL. Here are four queries, in Python's `sqlite3` so that nothing else is needed. ```python import json, sqlite3 db = sqlite3.connect("lab.db") # Sentences the converter left below F2. Receipts can raise them; see below. for row in db.execute("SELECT id, fidelity FROM nodes WHERE doc_key = 'hale2026harmonic' " "AND kind = 'sentence' AND fidelity IN ('F0', 'F1') ORDER BY seq"): print(row) # Every sentence that cites reference 2: the citation graph is sentence-level. for row in db.execute("SELECT c.sentence FROM citations c JOIN refs r " "ON r.doc_key = c.doc_key AND r.node = c.ref_node " "WHERE c.doc_key = 'hale2026harmonic' AND r.label = '2'"): print(row) # Who signed what. for row in db.execute("SELECT r.signer, r.scope, COUNT(*) FROM receipts r " "JOIN receipt_nodes rn ON rn.receipt = r.id GROUP BY r.id ORDER BY r.made_at"): print(row) # Table 1's body, row by row. rows = db.execute("SELECT r.id FROM nodes r JOIN nodes b ON b.doc_key = r.doc_key AND b.id = r.parent " "WHERE r.doc_key = 'hale2026harmonic' AND r.kind = 'row' AND b.kind = 'table-body' " "ORDER BY r.seq").fetchall() for (row,) in rows: cells = db.execute("SELECT content FROM nodes WHERE doc_key = 'hale2026harmonic' AND parent = ? ORDER BY ord", (row,)).fetchall() print(" | ".join("".join(r.get("text", "") for r in json.loads(c)) for (c,) in cells)) ``` ```text ('e4fkseg', 'F1') ('r1gacps', 'F0') ('eajq93t', 'F0') ('f929eq9',) ('ana', 'targeted:lode', 3) ('ana', 'targeted:lode', 23) ('ana', 'section:bzqvbhd', 2) ('cara', 'page:2', 3) ('ana', 'targeted:captions', 3) ('ben', 'qa:eqgbbpwb6725v', 10) ('ben', 'qa:gena76nkb6cnc', 3) Bond parameters CT–CT | 310.0 | 1.526 | 305.2 | 1.5260 CT–HC | 340.0 | 1.090 | 338.7 | 1.0900 OW–HW | 553.0 | 0.9572 | 548.1 | 0.9570 ``` The first query reads the level the converter recorded, not the final one. Receipts by `ana` raise `e4fkseg` and `r1gacps` to F3; `eajq93t` stays at F0 because the only receipt that covered it was revoked. Computing the final level means checking each receipt's signature, revocations and spot-checks, as [chapter 8](../spec/08-fidelity.md#82-levels) describes. ## Test your implementation against a vector Every vector file has one case per line, with its inputs and expected outputs. A harness for the projections looks like this, with your functions in place of the comments: ```python import json for line in open("vectors/projections.jsonl", encoding="utf-8"): case = json.loads(line) runs = case["runs"] # assert my_plain(runs) == case["plain"] # assert my_strict(runs) == case["strict"] # assert my_lossy(runs) == case["lossy"] # assert my_sha256(case["strict"]) == case["match_hash"] print(case["id"], case["strict"]) ``` [vectors/README.md](../vectors/README.md) lists every file and what it tests. ## Break something on purpose Change one digit in a fixture and run the checks: ```sh sed -i 's/1718 cm/1719 cm/' fixtures/packages/okafor2025open/nodes.jsonl pixi run quick ``` The checker reports the package's manifest hash no longer matches, and that unloading it from the database no longer gives the same bytes. Restore the file with `git checkout fixtures/packages/okafor2025open/nodes.jsonl`. The broken packages in [fixtures/invalid/](../fixtures/invalid/) are this kind of change, each with the rule it breaks.