infra/paperspec
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: documents and session tokens
check / check (default, macos-latest) (push) Canceled after 0s
Details
check / check (default, ubuntu-latest) (push) Canceled after 0s
Details
check / check (py310, macos-latest) (push) Canceled after 0s
Details
check / check (py310, ubuntu-latest) (push) Canceled after 0s
Details

Browse Source
This commit is contained in:
alexm committed 2026-10-12 01:07:50 -04:00
1 parent 62f54ed869
commit 8b2df58266
39 files changed
+1523 -199
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/README.md
+17
View File
@@ -0,0 +1,17 @@
# Guides
These guides explain the project to someone meeting it for the first time.
The [specification](../spec/README.md) says exactly what each format is; the guides say what it is for, with examples drawn from the fixtures.
| Guide | For |
| --- | --- |
| [Overview](overview.md) | What the project is, who it serves, and how its parts fit together. |
| [Concepts](concepts.md) | The ideas behind the formats, one at a time, each with a real example. |
| [Walkthrough](walkthrough.md) | One paper, from a flawed first reading to a corrected, reviewed and signed document. |
| [Examples](examples.md) | Things to run: checking hashes by hand, reading a package with a few lines of Python, querying a database. |
| [Rationale](rationale.md) | Why the formats look the way they do, and the questions they leave open. |
| [FAQ](faq.md) | Short answers to the questions people ask first. |
| [Glossary](glossary.md) | Every term, in a sentence. |
A good order for a first afternoon is the overview, then the walkthrough with the fixture open beside it, then the examples.
Read the concepts when a term in the walkthrough needs more than the glossary gives.
docs/concepts.md
+262
View File
@@ -0,0 +1,262 @@
# Concepts
Each section introduces one idea, shows it in the example paper `hale2026harmonic` (a synthetic chemistry paper in [fixtures/packages/](../fixtures/packages/)), and links to the chapter that defines it exactly.
## Works, versions, witnesses and documents
Four things are easy to confuse, so the formats keep them apart.
| Object | In the example |
| --- | --- |
| A **work** is the paper across all its versions. | Hale and Osei's paper on harmonic bond parameters, `doi:10.5555/example.hale.2026`. |
| A **version** is one expression of it: a preprint, the version of record, an erratum, the Supporting Information. | The version of record. |
| A **witness** is one file of a version, identified by its SHA-256. | The publisher's PDF, witness `qptsanv`. |
| A **document** is the text established for one version, from one or more witnesses. | `hale2026harmonic`. |
A document's key is its work's key, with a suffix for anything but the version of record: `hale2026harmonic.si` would be its Supporting Information.
See [chapter 2](../spec/02-model.md).
## A document is a tree of nodes
Every part of the text is a node: the document, its sections, headings, paragraphs and sentences, figures and their panels and captions, tables with their rows and cells, equations, footnotes and references.
The example has 75 nodes; its outline is:
```text
bzqvbhd 1 Introduction
qn08tj9 equation (1)
f0tj6ca 2 Methods
ajfvq4s figure Figure 1
gm5z4rx table Table 1
```
`nodes.jsonl` holds one node per line, in reading order, each with its kind, parent, position, content and hashes.
A sentence's content is a list of **runs**: plain text, styled text, inline math, citations, cross-references, footnote marks, links, and display equations the sentence runs through.
This is one table cell, shortened:
```json
{"id":"nkm3xn5","kind":"cell","parent":"zhe4g0k","ord":1,
"attrs":{"column":"…","role":"data","rowspan":1,"colspan":1},
"content":[{"kind":"text","text":"310.0"}],
"fidelity":"F1","text_hash":"…","match_hash":"…","align_key":"…","own_hash":"…","node_hash":"21f13b…"}
```
See [chapter 3](../spec/03-nodes.md).
## IDs are names, not positions
`nkm3xn5` says nothing about where the cell is.
That is deliberate: an ID that encoded a page or an ordinal would change when a better converter ordered blocks differently.
IDs are minted once from the document key, the node's kind, its text and an occurrence count, so a fresh conversion of the same input gives the same IDs, and then they never change.
When two nodes merge, one ID is retired and points at its successor; when an artifact is deleted, its ID is retired with no successor.
Legacy IDs, such as the block IDs older tools used, live on as aliases: in the example, `b-p01-text-003` still leads to sentence `f929eq9`.
See [chapter 4](../spec/04-identity.md).
## Addresses
An address names a node, and optionally pins it to a state and narrows it to a span:
| Address | Names |
| --- | --- |
| `hale2026harmonic#f929eq9` | The sentence about water, whatever it says now. |
| `hale2026harmonic#f929eq9@cd6cdb` | That sentence as it read when its `node_hash` began `cd6cdb`. |
| `hale2026harmonic#f929eq9@cd6cdb[29:44]` | Code points 29 to 44 of it in that state: "first test case". |
| `smithlab/hale2026harmonic#f929eq9` | The same sentence in the `smithlab` store. |
On the web the ID is the fragment, so a link to `…/smithlab/hale2026harmonic#f929eq9` lands on the sentence.
See [chapter 6](../spec/06-anchors.md#61-addresses).
## Five hashes, five jobs
Every node carries five content values.
| Value | Covers | Answers |
| --- | --- | --- |
| `text_hash` | The text exactly, normalized. | Is this the same quotation, character for character? |
| `match_hash` | The text with notation folded to one form. | Do two witnesses really agree? |
| `align_key` | The text folded loosely. | Is this probably the same sentence? |
| `own_hash` | The node's own content, without its children. | Has this section itself changed, as opposed to something inside it? |
| `node_hash` | The node and everything below it. | Has anything here changed at all? |
The middle two exist because one hash cannot do both of their jobs.
Chemistry writes the same thing many ways, and every one of these sentences has the same `match_hash`:
| As stored | Strict projection |
| --- | --- |
| `The solvent is $H_2O$.` | `The solvent is H_{2}O.` |
| `The solvent is H$_2$O.` | `The solvent is H_{2}O.` |
| `The solvent is H` and a subscript `2`, then `O.` | `The solvent is H_{2}O.` |
| `The solvent is H₂O.` | `The solvent is H_{2}O.` |
| `The solvent is $H_2 O$.` in Typst | `The solvent is H_{2}O.` |
But OCR often drops a superscript, and "103" must never be certified as agreeing with 10³.
So "A rate of 103 per second." and "A rate of 10³ per second." share an `align_key`, which proposes that they are the same sentence, while their `match_hash` values differ, which raises a suspect of kind `number` for a person to resolve.
The same care keeps "The energy" followed by citation 3 from matching the text "The energy3".
See [chapter 5](../spec/05-hashes.md) and [vectors/projections.jsonl](../vectors/projections.jsonl).
## Anchors
An anchor is how anything points into a document: a Lode extract, an annotation, a dataset value, a draft's citation.
It is a pinned address with enough context to find its text again after the document changes:
```json
{"format":"scienting-anchor/1","store":"smithlab","doc":"hale2026harmonic","node":"f929eq9","kind":"sentence",
"own_hash":"d5b2eb…","node_hash":"cd6cdb…","span":[29,44],
"quote":{"exact":"first test case","prefix":"Water, H$_2$O, served as the ","suffix":", with a bending band near 1595 "},
"rev":12}
```
`rev` is the document's revision when the anchor was made, counted in corrections rather than clock time.
When the document changes, an anchor resolves one of five ways:
| Result | Means | In the example |
| --- | --- | --- |
| `current` | Nothing it points at has changed. | A note on the water sentence. |
| `changed-below` | A container is unchanged in itself, but something inside it changed; a drift figure says how much. | A note on Table 1 after a row was inserted. |
| `moved` | Its node was merged into another that still holds the quote. | A note on half of a sentence that was wrongly split, then rejoined. |
| `reattached` | Its node changed, but the quote is still there exactly once, so it is re-pinned. | A note on "treated as zero" in a sentence whose superscript was fixed. |
| `stale` | It can no longer be trusted; a person must re-anchor it. | A note on the cell that read "3l0.0" before it was corrected to "310.0". |
See [chapter 6](../spec/06-anchors.md); the [walkthrough](walkthrough.md#5-what-happened-to-the-notes) shows all five.
## Corrections
A document is never edited in place.
Every fix is a correction: an operation with its evidence, its author and a reason, appended to `corrections.jsonl`.
```json
{"format":"scienting-correction/1","index":1,"op":"replace","target":"nkm3xn5",
"find":"3l0.0","replace":"310.0",
"evidence":[{"kind":"crop","sha256":"9b6fe6…"}],"author":{"key":"ana"},
"reason":"OCR read the digit 1 as the letter l.","at":"2026-10-02T10:00:00Z",
"before":"2fd1e9…","after":"21f13b…","touched":["nkm3xn5"]}
```
The operations are `replace`, `insert`, `delete`, `split`, `merge`, `move`, `reclassify`, `relabel` and `set`.
A gate refuses any correction that would change a digit without a crop of the page as evidence, or that would let a digit change anywhere it did not declare.
See [chapter 11](../spec/11-corrections.md).
## Fidelity
Every leaf has a level that says how far its text was checked.
| Level | Means |
| --- | --- |
| F0 converted | One reading, not corroborated. |
| F1 resolved | Every doubt about it was answered against the page image. |
| F2 corroborated | An independent witness agrees exactly, or the text was born digital. |
| F3 proofread | A named person compared it with the page and signed for its current text. |
A level counts only at the node's current hash, so a correction drops a proofread sentence back until someone signs for it again.
The example reports 28 leaves at F3, 12 at F2, 4 at F1 and 1 at F0.
See [chapter 8](../spec/08-fidelity.md).
## Receipts and spot-checks
Nobody signs three thousand nodes one at a time.
A reviewer signs a **receipt**: one signature over a Merkle root of everything reviewed in a sitting, with the evidence looked at.
In the example, the reviewer `ana` signs one receipt for 23 nodes: Table 1's cells, the equation and the sentences around it.
Signing once per sitting invites rubber-stamping, so a second reviewer spot-checks a sample.
The sample is chosen by the checker's Ed25519 signature over the receipt, which neither reviewer can steer, and anyone can recompute it.
Across samples, the record states a bound on each reviewer's error rate: zero errors in 10 sampled nodes bounds the rate below 26%, and zero errors in 300 bounds it below 1%.
A spot-check that finds an error suspends the receipt; a revocation withdraws it.
Either way the nodes fall back to the level they had without it.
See [chapter 8](../spec/08-fidelity.md#83-receipts).
## Page fingerprints
Publishers stamp each download with the institution and time, so two teammates' copies of one paper rarely share a SHA-256.
Each page therefore also has a fingerprint: a hash of its size, its text with the stamp lines removed, and its images.
Two downloads of the same page stamped "Downloaded via UNIVERSITY OF EXAMPLE on October 11, 2026" and "Downloaded via EXAMPLE PHARMA INC on March 2, 2027" share the fingerprint `6272e6f3…`; change one digit of the text and it becomes `b545d285…`.
Geometry and digitizations point at fingerprints, so a re-download keeps every box and every value read off a plot.
See [chapter 7](../spec/07-geometry.md).
## Keeping text private
A team may annotate a paper it may not share.
A **checks-only** package keeps every ID, label, page and box, but withholds the text, and keys every hash, because a plain hash of a short text such as "310.0" is recovered by hashing guesses.
Inside the team, values are plain SHA-256; exported to another of the team's machines, they are HMACs under the team's key; in a public layer, the key is derived from the paper's page fingerprints, so only someone holding the paper can check them.
[fixtures/packages/hale2026harmonic.checks-only](../fixtures/packages/hale2026harmonic.checks-only) is the example paper exported this way.
See [chapter 5.8](../spec/05-hashes.md#58-keyed-hashes).
## Rights
What a team may do with a paper depends on how it holds the paper.
Every witness carries a rights record: the license as stated, the basis (an open license, a subscription, a collective license, the team's own work), and which uses that allows, including AI uses by purpose and by where the processing runs.
Every processing stage declares what it needs and runs only when the rights allow it.
The example PDF is held on a subscription that allows text mining and internal AI use on the team's own machines.
So conversion by a local model is allowed, conversion by an outside service is refused for lacking `ai/internal/outside`, and model review in the team's cloud is refused for lacking `ai/internal/tenancy?` (the question mark means unknown) until a project owner signs a policy that settles it.
See [chapter 14](../spec/14-rights.md).
## Layers and annotations
Everything people and models add is an annotation, grouped in layers such as "Novice glosses" or "Journal club, March".
Annotations are W3C Web Annotations whose selector is an anchor, so any Web Annotation tool can read them.
This gloss, shortened, explains a phrase for a first-year student:
```json
{"type":"Annotation","motivation":"describing","status":"confirmed",
"body":{"type":"TextualBody","kind":"gloss","audience":"novice",
"value":"A *harmonic* term treats a bond like a spring: the energy grows with the square of the stretch."},
"target":{"type":"SpecificResource","source":"urn:scienting:doc:smithlab/hale2026harmonic",
"selector":[{"type":"AnchorSelector","anchor":{"node":"sxgcc9h","span":[111,130],"…":"…"}},
{"type":"TextQuoteSelector","exact":"bond force constant"}]}}
```
One rule decides what is a claim and what is a layer: a statement true independently of the paper belongs in Lode as a checked claim; a statement about reading the paper belongs in a layer.
Model-written text is always a `generated` annotation, never source.
See [chapter 18](../spec/18-layers.md).
## Exercises
Because every sentence and cell has an address, the answer to "find the evidence" is an address, and an address can be graded.
An exercise asks a student to find the cell giving the Set A bond constant for CT–CT; a student who points at `nkm3xn5` scores 1 of 1 with "That is the cell.", and one who points at the Set B cell scores 0 with "Look in Table 1, row CT–CT."
Students submit under pseudonyms, and a key whose anchors have gone stale is never used to grade.
See [chapter 19](../spec/19-exercises.md).
## Datasets, digitizations and views
Numbers live in prose, in tables and inside figures; all three should end up as data a team can plot and cite.
A dataset is a Frictionless Data Package with units, uncertainty, and an origin for every value.
The example's Table 1 becomes a CSV whose every value is anchored to the cell that prints it, digits exactly as printed:
```text
key,bond,set,k_r,r_eq
ct-ct-a,CT–CT,A,310.0,1.526
ct-ct-b,CT–CT,B,305.2,1.5260
```
A digitization records how values were read off a plot (axes, calibration, series, uncertainty) and counts only once a person confirms it against an overlay.
A view is a Vega-Lite specification that names its data by dataset ID and hash.
See chapters [15](../spec/15-datasets.md), [16](../spec/16-digitization.md) and [17](../spec/17-views.md).
## Packages, databases and bundles
A document travels as a package: a folder of canonical JSON files.
```text
hale2026harmonic/
MANIFEST.json every other file, with its SHA-256; its own hash pins the package
document.json the work, version, metadata, policy, root hash and fidelity
nodes.jsonl one node per line, in reading order
witnesses.json each witness and its rights
pages.jsonl each page and its fingerprint
locations.jsonl where each node is on each page
references.jsonl parsed references
corrections.jsonl every correction, in order
receipts.jsonl receipts and revocations
ids.jsonl retired IDs and legacy aliases
roster.json the keys that signed
provenance.json tools, versions and where each stage ran
assets/ images, by SHA-256
datasets/ datasets reported from the document
derived/ MathML and other renderings, rebuildable
```
Every line is canonical JSON, so a corrected sentence is a one-line diff, and every file has exactly one valid byte sequence.
The same data loads into tables that run on SQLite, Dolt and PostgreSQL, and unloads back to the same bytes.
A bundle is one SQLite file with every table and a full-text index, which a reader or an MCP server can query offline.
See chapters [12](../spec/12-package.md) and [13](../spec/13-database.md).
docs/examples.md
+221
View File
@@ -0,0 +1,221 @@
# Examples
Things to run, each with the output it gives.
Run them from the repository's root.
With [pixi](https://pixi.sh), `pixi run <task>` sets up the environment first; every command also works with plain Python 3.10 or later and standard command-line tools.
## Run the checks
```sh
pixi run check
```
`pixi run check` lints the Markdown and then runs `python tools/check.py`, which you can also run directly.
Each section prints how many checks it ran, and the last line is `All checks pass.`
`pixi run quick`, or `--quick`, skips the slowest part: rebuilding everything and comparing it byte for byte.
## Recompute a hash by hand
A sentence's `text_hash` is SHA-256 of its text, after Unicode NFC and with whitespace collapsed.
For a sentence of plain text, that is a one-liner:
```sh
printf '%s' 'The two parameter sets agree to within 2% for every bond.' | sha256sum
```
```text
175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a -
```
The same value is in the package:
```sh
grep -o '"id":"j06tfhn".*' fixtures/packages/hale2026harmonic/nodes.jsonl | grep -o '"text_hash":"[0-9a-f]*"'
```
```text
"text_hash":"175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a"
```
## Check a package's manifest
Every file of a package is listed in `MANIFEST.json` with its SHA-256, and the manifest's own hash pins the whole package.
```sh
cd fixtures/packages/okafor2025open
sha256sum nodes.jsonl
grep -o '"path":"nodes.jsonl","sha256":"[0-9a-f]*"' MANIFEST.json
```
```text
0fccd6492121fb91ebd1d32487e23ca5b30ad92627e7d3a0801d77549e1a2f94 nodes.jsonl
"path":"nodes.jsonl","sha256":"0fccd6492121fb91ebd1d32487e23ca5b30ad92627e7d3a0801d77549e1a2f94"
```
## Read a package with nothing but a JSON parser
Packages are plain JSON Lines.
Save this script as `outline.py`; it prints a document's sections, figures, tables and equations.
```python
import json, sys
pkg = sys.argv[1]
nodes = [json.loads(line) for line in open(pkg + "/nodes.jsonl", encoding="utf-8")]
by_id = {n["id"]: n for n in nodes}
children = {}
for n in nodes:
children.setdefault(n["parent"], []).append(n)
def text(n):
"""The plain projection of a node with runs."""
out = []
for r in n["content"]:
if r["kind"] == "math":
out.append("$" + r["source"] + "$")
elif r["kind"] == "equation":
out.append("$$" + by_id[r["node"]]["content"]["source"] + "$$")
else:
out.append(r["text"])
return "".join(out)
for n in nodes:
if n["kind"] == "section":
heading = children[n["id"]][0]
print(n["id"], n.get("label", ""), " ".join(text(s) for s in children[heading["id"]]))
elif n["kind"] in ("figure", "table", "equation"):
print(n["id"], " ", n["kind"], n.get("label", ""))
```
```sh
python3 outline.py fixtures/packages/hale2026harmonic
```
```text
bzqvbhd 1 Introduction
qn08tj9 equation (1)
f0tj6ca 2 Methods
ajfvq4s figure Figure 1
gm5z4rx table Table 1
```
## Follow an address
`hale2026harmonic#f929eq9[29:44]` names code points 29 to 44 of sentence `f929eq9`.
With `text()` from the script above:
```python
print(text(by_id["f929eq9"])[29:44])
```
```text
first test case
```
Offsets count Unicode code points; in Python, string indexes already do.
In JavaScript, convert to UTF-16 units first; [vectors/offsets.jsonl](../vectors/offsets.jsonl) gives both for text with characters outside the Basic Multilingual Plane.
## Load packages into SQLite
```sh
pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic fixtures/packages/okafor2025open fixtures/packages/smithlab2027charges
```
Without pixi, run `python3 tools/load.py` with the same arguments.
```text
hale2026harmonic: 75 nodes
okafor2025open: 15 nodes
smithlab2027charges: 15 nodes
```
The tables are those of [sql/doc.sql](../sql/doc.sql) and [sql/store.sql](../sql/store.sql), and the same statements run on Dolt and PostgreSQL.
Here are four queries, in Python's `sqlite3` so that nothing else is needed.
```python
import json, sqlite3
db = sqlite3.connect("lab.db")
# Sentences the converter left below F2. Receipts can raise them; see below.
for row in db.execute("SELECT id, fidelity FROM nodes WHERE doc_key = 'hale2026harmonic' "
"AND kind = 'sentence' AND fidelity IN ('F0', 'F1') ORDER BY seq"):
print(row)
# Every sentence that cites reference 2: the citation graph is sentence-level.
for row in db.execute("SELECT c.sentence FROM citations c JOIN refs r "
"ON r.doc_key = c.doc_key AND r.node = c.ref_node "
"WHERE c.doc_key = 'hale2026harmonic' AND r.label = '2'"):
print(row)
# Who signed what.
for row in db.execute("SELECT r.signer, r.scope, COUNT(*) FROM receipts r "
"JOIN receipt_nodes rn ON rn.receipt = r.id GROUP BY r.id ORDER BY r.made_at"):
print(row)
# Table 1's body, row by row.
rows = db.execute("SELECT r.id FROM nodes r JOIN nodes b ON b.doc_key = r.doc_key AND b.id = r.parent "
"WHERE r.doc_key = 'hale2026harmonic' AND r.kind = 'row' AND b.kind = 'table-body' "
"ORDER BY r.seq").fetchall()
for (row,) in rows:
cells = db.execute("SELECT content FROM nodes WHERE doc_key = 'hale2026harmonic' AND parent = ? ORDER BY ord",
(row,)).fetchall()
print(" | ".join("".join(r.get("text", "") for r in json.loads(c)) for (c,) in cells))
```
```text
('e4fkseg', 'F1')
('r1gacps', 'F0')
('eajq93t', 'F0')
('f929eq9',)
('ana', 'targeted:lode', 3)
('ana', 'targeted:lode', 23)
('ana', 'section:bzqvbhd', 2)
('cara', 'page:2', 3)
('ana', 'targeted:captions', 3)
('ben', 'qa:eqgbbpwb6725v', 10)
('ben', 'qa:gena76nkb6cnc', 3)
Bond parameters
CT–CT | 310.0 | 1.526 | 305.2 | 1.5260
CT–HC | 340.0 | 1.090 | 338.7 | 1.0900
OW–HW | 553.0 | 0.9572 | 548.1 | 0.9570
```
The first query reads the level the converter recorded, not the final one.
Receipts by `ana` raise `e4fkseg` and `r1gacps` to F3; `eajq93t` stays at F0 because the only receipt that covered it was revoked.
Computing the final level means checking each receipt's signature, revocations and spot-checks, as [chapter 8](../spec/08-fidelity.md#82-levels) describes.
## Test your implementation against a vector
Every vector file has one case per line, with its inputs and expected outputs.
A harness for the projections looks like this, with your functions in place of the comments:
```python
import json
for line in open("vectors/projections.jsonl", encoding="utf-8"):
case = json.loads(line)
runs = case["runs"]
# assert my_plain(runs) == case["plain"]
# assert my_strict(runs) == case["strict"]
# assert my_lossy(runs) == case["lossy"]
# assert my_sha256(case["strict"]) == case["match_hash"]
print(case["id"], case["strict"])
```
[vectors/README.md](../vectors/README.md) lists every file and what it tests.
## Break something on purpose
Change one digit in a fixture and run the checks:
```sh
sed -i 's/1718 cm/1719 cm/' fixtures/packages/okafor2025open/nodes.jsonl
pixi run quick
```
The checker reports the package's manifest hash no longer matches, and that unloading it from the database no longer gives the same bytes.
Restore the file with `git checkout fixtures/packages/okafor2025open/nodes.jsonl`.
The broken packages in [fixtures/invalid/](../fixtures/invalid/) are this kind of change, each with the rule it breaks.
docs/faq.md
+45
View File
@@ -0,0 +1,45 @@
# FAQ
**Can I read these files without the studio's software?**
Yes.
Every file is JSON or JSON Lines, every format is specified in [spec/](../spec/README.md), and [the examples](examples.md) read a package with a few lines of Python.
**Why not keep using PDFs and page numbers?**
A page number names a region, not a statement, and it changes between a preprint and the version of record.
An address names a sentence or a cell, survives reconversion, and can be checked against the exact text.
**Why are numbers stored as strings?**
Because databases and JSON libraries rewrite numbers: `310.0` becomes `310` and `1.5260` becomes `1.526`, and the printed digits are part of what a paper says.
**What happens when a better converter comes along?**
Its reading is aligned with the established document sentence by sentence.
Sentences it reads the same keep their IDs; every difference becomes a suspect for review; nothing changes until a correction passes the gate.
**Do I need receipts, rights records and spot-checks for a personal library?**
No.
One person can leave receipts out entirely and record only a license.
The fields are defined so that a personal library loads into a team's store later without conversion.
**Is a model allowed to correct a document?**
A model may propose corrections and answer suspects in a review session, under the same gate as a person, and its corrections record the model and the run.
Only a named person can sign a receipt.
**Why are annotations W3C Web Annotations?**
So existing tools can read them.
A tool that does not know the anchor selector falls back to the text-quote selector beside it.
**What is the difference between a claim and an annotation?**
A statement true independently of the paper, such as "K_r is the bond force constant of the AMBER energy", is a claim and belongs in Lode, where it is checked.
A statement about reading the paper, such as "the authors pass quickly over the scaling here", is an annotation in a layer, attributed but not checked.
**Can a team share notes on a paywalled paper?**
Yes, without sharing the paper: a checks-only package or a public layer carries IDs, offsets and keyed hashes, and is shown over each reader's own copy.
Whether a team may do so is a question for its own counsel; nothing here is legal advice.
**How do I add a node kind or a run kind?**
Propose it with the vector or fixture that shows it; a new kind is a minor version.
See [CONTRIBUTING.md](../CONTRIBUTING.md).
**What are the files in `tools/` for?**
They compute and check every vector and fixture so that no expected value is typed by hand.
They are not a library; implementations follow the specification.
docs/glossary.md
+118
View File
@@ -0,0 +1,118 @@
# Glossary
Address
: A string naming a node, optionally pinned to a hash and narrowed to a span: `key#id@hash6[start:end]`.
Alias
: A legacy ID kept so that old references still lead to the node that now holds their text.
Align key
: A hash of a node's lossy projection, used only to propose that two readings hold the same sentence.
Anchor
: The stored form of a pinned address, with its quotation and context, that resolves as current, changed below, moved, reattached or stale.
Asset store
: Where a team keeps files by SHA-256: witnesses, images, crops and withheld text.
Bundle
: One SQLite file with every table and a full-text index, which readers, servers and MCP servers query.
Changed below
: The resolution of an anchor on a container that is unchanged in itself while something inside it changed.
Correction
: An operation on an established document, with its evidence and author; never edited, only appended.
Document
: The text established for one version of one work, as a tree of nodes.
Drift
: How much has changed below a container since an anchor on it was made, counted from the corrections.
Exercise
: An annotation whose body holds a prompt, an answer key of anchors or a value, and a grading rule.
Fidelity level
: How far a leaf's text was checked against its source, from F0 converted to F3 proofread.
Keyed hash
: A content value computed as an HMAC under a team key or a key derived from page fingerprints, so a short text cannot be recovered by hashing guesses.
Layer
: A named set of annotations with one purpose, audience, visibility and license.
Leaf
: A node that holds text itself: a sentence, cell, reference, author line, keyword line, equation, equation line, code or algorithm.
Lossy projection
: A node's text with scripts flattened, citation marks, whitespace and dashes removed, and case folded; used for proposing alignments.
Match hash
: A hash of a node's strict projection; equal match hashes are what corroboration requires.
Node
: An addressable part of a document: a section, paragraph, sentence, figure, panel, caption, table, row, cell, equation, footnote, reference and the rest.
Own hash
: A hash of a node's own content without its children, used to judge anchors on containers.
Package
: A document as a folder of canonical JSON files with a manifest whose hash pins them all.
Page fingerprint
: A hash of a page's size, its text with download stamps removed, and its images, so that differently stamped downloads share their pages.
Plain projection
: A node's text as one string, on which offsets, quotations and text hashes are defined.
Policy
: A project owner's signed decision that settles rights a record leaves unknown.
Project
: A team's curated set of documents, with its members, layers, data, reading paths and drafts.
Receipt
: One signature over a Merkle root of what a person reviewed in a sitting, which sets the level of what it lists.
Revision
: The index of the last correction applied to a document; anchors and receipts record it.
Revocation
: A signed record that withdraws a receipt, or every receipt by one signer in a period.
Rights record
: A witness's license, the basis on which the team holds it, and the uses that basis allows.
Roster
: The list of keys that may sign, with their owners and purposes.
Spot-check
: A second reviewer's check of a sample of a receipt's nodes, drawn from a seed nobody can steer.
Stamp rule
: A pattern that recognizes a publisher's download stamp in a page's text, so that fingerprints ignore it.
Strict projection
: A node's text with notation folded to one canonical form and structure kept; used for certifying agreement.
Suspect
: A doubtful reading raised for a person or a review session to answer.
Targeted F3
: Proofreading the nodes a team's claims, quotations and datasets rest on, rather than every node.
Text policy
: Whether a package carries its text (`full`) or withholds it (`checks-only`).
Verdict
: A judgment of whether a cited passage supports a citing sentence: supports, partly, does not, or cannot tell.
Version
: One expression of a work: a preprint, the version of record, an erratum, the Supporting Information.
Witness
: One file of one version of a work, identified by its SHA-256.
Work
: A piece of scholarship across all its versions.
docs/overview.md
+92
View File
@@ -0,0 +1,92 @@
# Overview
## The idea
Most tools for scientific literature aggregate: they index hundreds of millions of papers and answer questions over all of them, shallowly.
This project does the opposite.
A team working on a focused question cares more about fifty papers it understands completely than about the world's literature.
Those fifty papers deserve to be converted once, corrected once, split down to the sentence, annotated by the people who know them, linked to checked claims, and kept.
What that work produces answers two questions scientists ask every day.
1. **Does the cited passage say what is claimed?**
A citation to a real paper can still be inaccurate, incomplete or beside the point, and a reader needs to see the exact sentence it rests on.
2. **Where did this number come from, and how far was it checked?**
A force constant in a table, a value read off a plot, a parameter in a model: each should lead back to the cell or sentence it came from, with a record of who compared it with the page.
A model can summarize a paper, but it cannot produce a signed statement that a person checked a passage, IDs that never move, or a store its owner controls.
Those are what these formats are for, and they are worth most at the scale of one team.
## Who it is for
| User | Example |
| --- | --- |
| One researcher | A thesis student, a science writer or fact-checker, a Wikipedia editor checking a citation, a scientist keeping a deep personal library. |
| An academic lab | A research group, a thesis committee, a journal club reading one annotated paper together. |
| A course | An instructor whose students learn to check whether a source supports a claim, with answers that are addresses and can be graded. |
| An industrial team | A computational chemistry or materials group that needs to show which passage of which paper each parameter came from. |
"Team" throughout means roughly five to twenty people with a few dozen to a few hundred papers that matter.
Everything is designed for that scale: hundreds of documents and millions of sentences, not hundreds of millions of papers.
## The parts
The formats in this repository are the meeting point of several programs.
No program imports another's code; they meet only through these files.
```text
spec (this repository)
│
┌─────────────────┼──────────────────┐
core core-ts typst-scienting
┌──────┬──┴─────┬────────┐ │
paperpack digitize lode varve varve-web
```
| Part | Does |
| --- | --- |
| **spec** | Defines every format: prose, schemas, tables, test vectors and fixtures. |
| **core** | `scienting-core`, the reference implementation of the shared rules in Python, standard library only: canonical JSON, hashing, IDs, addresses, anchors, packages, receipts and rights checks. |
| **core-ts** | The same rules in TypeScript, for the browser. |
| **paperpack** | The converter. It turns a PDF, Supporting Information, JATS XML, LaTeX, HTML, EPUB or a Typst project into a package, raises doubtful readings as suspects, and applies corrections only when they pass its gate. |
| **digitize** | Reads values off plots: calibrated axes, traced series, an overlay a person confirms. |
| **Lode** | The store of checked claims. Its extracts point at sentences through anchors, so a claim can verify itself against the words it rests on. |
| **Varve** | The team workspace: a store a team can clone, a server, a reader, an editor and an MCP server that lets a team's own models quote the corpus by address. |
| **typst-scienting** | A Typst package that lets a draft cite sentences, claims and datasets. |
A varve is one year's pair of sediment layers on a lake bed; geologists count them because each layer is laid down once and kept.
That is the product's idea: every reading a team does leaves a layer, dated and signed, over a text that never has to be redone.
## How a paper moves through the system
1. **Acquire.** A witness arrives (the publisher's PDF, say) with a rights record that says how the team holds it and what that allows.
2. **Read and reconcile.** paperpack reads it with one or more backends, compares the readings with each other and with the PDF's text layer, and fingerprints every page so that differently stamped downloads are recognized as the same pages.
3. **Segment and mint.** Text is split into sentences, and every section, paragraph, sentence, figure, panel, table, row, cell, equation, footnote and reference gets a permanent ID and five content hashes.
4. **Review.** Disagreements become suspects with page crops; a person or a model session answers them as corrections, and the gate refuses any correction that changes a digit without image evidence.
5. **Package.** The result is a package: a folder of canonical JSON files with a manifest whose hash pins the whole.
6. **Use.** The package loads into a database, a reader renders it with layers of annotations, Lode anchors claims to it, datasets anchor values to its cells, and drafts cite its sentences.
7. **Check and sign.** People proofread what matters and sign receipts; second reviewers spot-check them; every level counts only at the current text.
[The walkthrough](walkthrough.md) follows one paper through every step.
## Principles
| Principle | Means |
| --- | --- |
| Quality over quantity | A small corpus processed to a stated and checkable fidelity beats a large one processed to an unknown one. |
| Do the work once | Conversion, correction, segmentation and digitization are recorded and kept; nothing is reparsed to be fixed again. |
| Address everything | Every section, paragraph, sentence, figure, panel, caption, table, cell, equation, footnote and reference has a permanent ID and a content hash. |
| Keep source and commentary apart | What a source says is stored as the source said it; everything people or models add is stored beside it and points at it. |
| Count only what was done | A level says how far something was checked, and it falls back when the checked content changes. |
| Models propose, people sign | Models draft, segment, gloss and digitize; only a named person confirms. |
| The team owns its data | Every store is a database the team can clone, and every format is documented and readable without the studio's code. |
## What it is not
It is not a search engine over all literature, a citation index, a recommendation feed or a chat-with-any-PDF tool.
Tools may import from Crossref, OpenAlex, PubMed Central or arXiv, but never depend on them to work.
## Where next
- [Concepts](concepts.md) explains each idea with an example.
- [The specification](../spec/README.md) defines each format exactly.
docs/rationale.md
+150
View File
@@ -0,0 +1,150 @@
# Rationale
Why the formats look the way they do.
Each choice below is one a reader is likely to question; the specification states the rule, and this page gives the reason.
## Identity
**IDs carry no meaning.**
An ID that encodes a page, an ordinal or a kind changes when a better converter orders blocks differently, or when a paragraph turns out to be a caption.
An opaque name minted from the text survives both, and a fresh conversion of the same input still gets the same IDs.
**IDs never move, and retired IDs leave a trail.**
Everything a team builds points at IDs: notes, claims, dataset values, citations in drafts.
So a merge records its successor, a deletion records that there is none, and legacy IDs live on as aliases.
**Revisions are counted, not timed.**
Anchors are made on different machines and packages are imported later, and clocks disagree; a correction index does not.
## Hashes and projections
**Five values, not one.**
Quotations need every character; certifying that two witnesses agree needs notation folded but structure kept; proposing that two sentences are the same needs something looser still; and containers need a hash of themselves apart from what they hold.
No single hash does all four jobs.
**Only the strict projection certifies.**
OCR drops superscripts, so a projection loose enough to match "103" with 10³ would certify a thousandfold error.
The lossy key may propose that match; only the strict hash may accept it.
**Citation marks stay marked in the strict projection.**
Otherwise "10" followed by citation 3 would agree with the text "103".
**Decimal points and commas count as simple notation.**
Otherwise `$-1.5$` in a LaTeX source would be complex math and could never agree with "−1.5" in the PDF.
**A hyphen or en dash before a digit is a minus sign in the strict projection.**
Text layers and OCR disagree about dashes in "10–20" and "−5"; folding them before digits makes witnesses agree without touching "CT–CT".
**The lossy key deletes whitespace and dashes.**
OCR merges and splits words and line-break hyphens; a key that only proposes alignments should forgive that.
**Offsets index the stored text, and structural hashes cover it exactly.**
When `node_hash` is unchanged, every stored offset is still exact, and a renderer can slice the stored text without normalizing it first.
Only the three text values normalize.
**Hashes that leave the team are keyed.**
A plain hash of "310.0" is recovered by hashing guesses, and overlapping anchors can rebuild passages.
Inside a team, plain hashes are fine; anything exported is an HMAC under the team's key, or under a key derived from the paper's own pages, which only someone holding the paper can compute.
## Anchors
**Every anchor carries `own_hash`, and anchors on containers without content carry their leaves.**
Telling a relabel from a real change needs the anchored `own_hash`, and deciding whether a merged paragraph still holds "at least half" of what was anchored needs the original leaves.
**Long quotations become a hash and a length.**
Quotations are capped at 200 code points to keep anchors small and to limit how much text an anchor carries; a hashed window can still be found again in the node.
**Prefix and suffix pick between repeated quotes.**
"The value" may occur twice in a sentence; the context around it, as in W3C text-quote selectors, decides which one was meant.
**`changed-below` reports drift.**
A note on a section stays valid when a typo inside it is fixed, but not forever: once a large share of what is inside has changed, the note is flagged for review.
**A row's header path is not hashed.**
Rows have no content of their own, so inserting a dropped row-group header does not stale the anchors on every cell beneath it.
## Corrections
**Corrections are appended, never edited.**
A document's history is its first reading plus its corrections, so anyone can see what changed, when, why and on what evidence.
**There is a `set` operation.**
Readings drop spans, roles and header paths as often as they drop words, and those need a correction too.
**Applied corrections record what they touched.**
With `path`, `to_path` and `touched` in each record, drift is computed from the records alone, with no need to replay history.
**Digits need image evidence.**
A wrong digit is the most expensive error a scientific text can carry and the easiest for a model to introduce, so no digit changes without a crop of the page.
## Geometry
**Pages have fingerprints.**
Publishers stamp each download, so file hashes cannot identify a paper across teammates' copies; fingerprints with the stamps removed can.
**Fingerprint text deletes whitespace entirely.**
PDF text extractors disagree about where to insert spaces, and the fingerprint must come out the same in a converter and in a browser.
## Review
**People sign receipts, not nodes.**
Nobody signs three thousand times, so one signature covers a Merkle root of everything reviewed in a sitting.
**Spot-check samples come from a signature nobody can steer.**
A seed the reviewer could influence could be ground until the sample fell on careful work.
The checker's deterministic Ed25519 signature over the receipt is unpredictable to the reviewer and unchoosable by the checker, and anyone can recompute it.
**Error rates are reported as bounds.**
One clean sample of ten proves little; a Clopper–Pearson bound across all of a reviewer's samples says what can actually be claimed.
**Levels fall back.**
A level counts only at the current text, so a correction never leaves a stale "proofread" behind.
## Rights
**Rights are recorded per witness, not per paper.**
What a team may do depends on how it obtained the paper: a subscription, a mining license, an open license, its own work.
**AI uses have two axes.**
Reviewers ask not only whether a model may see the text, but where it runs and whether the text is kept there.
**Unknown means no.**
A stage whose rights are unknown does not run; a project owner settles the question with a signed policy, so the decision is the team's and is on the record.
A policy fills in unknowns and never overrides an explicit answer.
**Backend licenses are part of the gate.**
A converter's code and its model weights can be licensed differently, and the weights may exclude commercial or competing use.
## Packages and databases
**Canonical JSON, one record per line.**
A corrected sentence is a one-line diff, and every file has exactly one valid byte sequence, so a hash of the manifest pins the whole package.
**Checks-only packages drop everything that carries text.**
Corrections carry their find and replace strings, datasets carry values, figure text and reference records carry text; a checks-only export omits them all.
**SQL avoids reserved words in every engine.**
The same tables run on SQLite, Dolt (MySQL-compatible) and PostgreSQL, where `key`, `index`, `before` and `by` are reserved; hence `doc_key`, `idx`, `before_hash` and `signer`.
**Loading and unloading are exact inverses.**
A team can keep its documents in a database and still hand anyone the package, byte for byte.
## Questions the formats leave open
These are settled for now, but a future version may revisit them.
| Question | Current answer |
| --- | --- |
| Should node IDs carry a kind prefix, such as `s-k3m9q2a`, to help people debugging? | No: opaque IDs survive reclassification. |
| Should `text_hash` normalize to NFC, or only collapse whitespace as Lode's extracts do? | NFC and whitespace; existing Lode extracts keep Lode's rule. |
| How long may a quotation in an anchor be? | 200 code points. |
| How is complex math compared? | By its source, run by run; comparing canonical MathML is a possible later addition. |
| What extension and media type do zipped packages use? | `.sct.zip` and `application/vnd.scienting.doc+zip`. |
| Which drift threshold, spot-check rate and digit rule should teams use? | A quarter of a container's leaves; 5% of a receipt's nodes, at least ten; and `all`, `anchored-sections` or `none`, set per project. |
| How should rights combine when a document comes from witnesses held on different bases? | The most restrictive witness governs each use. |
## Limits worth knowing
- Notation drops spaces and text keeps them, so `$T = 300$` from a LaTeX source and "T = 300" from a PDF disagree, and become a `word` suspect for review.
- Page fingerprints are computed from extracted text lines; two programs that should agree on fingerprints must extract lines the same way.
- The roster is defined here and is meant to be the same one Lode verifies; the manifest is meant to have the shape of Lode's `lode-cache/1`.
docs/walkthrough.md
+126
View File
@@ -0,0 +1,126 @@
# Walkthrough: one paper, start to finish
This guide follows the example paper `hale2026harmonic` from a flawed first reading to a corrected, reviewed and signed document.
The paper is synthetic, written for the fixtures, but everything that happens to it happens to real papers.
Open [fixtures/packages/hale2026harmonic/](../fixtures/packages/hale2026harmonic/) beside this page: every ID below is in it.
The hand-written source of the first reading is [fixtures/src/hale2026harmonic.json](../fixtures/src/hale2026harmonic.json).
## 1. A PDF arrives
The team downloads the publisher's PDF.
It becomes a witness, identified by its SHA-256, with the short ID `qptsanv`, and a rights record: the library's subscription allows reading, text mining, and AI processing on the team's own machines, until August 31, 2027; whether it allows processing in the team's cloud is unknown.
The publisher stamped the download in the margin with the institution and the time.
Fingerprinting each page removes those lines first (the rules `downloaded-via` and `sharing-guidelines` fired on both pages), so page 101's fingerprint is `6272e6f3…` in this copy and in a colleague's download stamped a year later.
See [`pages.jsonl`](../fixtures/packages/hale2026harmonic/pages.jsonl).
## 2. The first reading, with its mistakes
A converter reads the PDF into sections, paragraphs, sentences, a figure, a table, an equation, a footnote and references, and mints a permanent ID for every node.
Like every reading, this one has mistakes:
| What the reading did | What the page says |
| --- | --- |
| A cell read `3l0.0`. | `310.0`, with the digit 1. |
| A sentence read `Values below 10-3 kcal mol⁻¹…`. | A superscript: 10⁻³. |
| Two sentences ran together: "…treated as zero. Table 1 lists…". | Two sentences. |
| One sentence was cut in two at a line break: "…and the energies were" and "fitted by least squares." | One sentence. |
| A paragraph was split at a column break. | One paragraph. |
| The running head "J. Example Chem. 2026, 14, 101–109" became a paragraph. | A running head, not text. |
| The equation's number read `(l)` and the figure's `Figure l`. | `(1)` and `Figure 1`. |
| The CT–HC row merged two cells into "338.7 1.0900". | Two cells. |
| The OW–HW row was missing. | It is there. |
| "Set B" spanned one column. | It spans two. |
The doubtful places become suspects with crops of the page, for a person to answer.
The converter does not guess, and nothing changes until a correction passes the gate.
## 3. Twelve corrections
Each fix is appended to [`corrections.jsonl`](../fixtures/packages/hale2026harmonic/corrections.jsonl), numbered, with its evidence, its author and its reason.
| # | Operation | On | What it did |
| --- | --- | --- | --- |
| 1 | `replace` | cell `nkm3xn5` | `3l0.0` to `310.0`. A digit changed, so the gate required a crop, and it has one. |
| 2 | `replace` | sentence `e4fkseg` | `10-3` to the runs `10` and superscript `−3`. |
| 3 | `split` | sentence `e4fkseg` | Split before "Table 1"; the first part keeps `e4fkseg`, the second is new: `r1gacps`. |
| 4 | `merge` | sentence `eajq93t` | Joined with `e9rh611`, which is retired with `eajq93t` as its successor. |
| 5 | `merge` | paragraph `zyqr0yc` | Joined with `g44k023`, retired the same way. |
| 6 | `delete` | paragraph `nm7vyzk` | The running head is gone; its IDs are retired with no successor. |
| 7 | `relabel` | equation `qn08tj9` | `(l)` to `(1)`. |
| 8 | `delete` | row `tfjge3c` | The garbled CT–HC row and its four cells. |
| 9 | `insert` | row `fbg2mwq` | The CT–HC row, read again: five new cells. |
| 10 | `insert` | row `j0x5twd` | The missing OW–HW row. |
| 11 | `set` | cell `vtszvw9` | Set B's `colspan` from 1 to 2. |
| 12 | `relabel` | figure `ajfvq4s` | `Figure l` to `Figure 1`. |
The document is now at revision 12.
Notice what did not happen: no other cell, sentence or row changed its ID or its hashes.
Inserting a row adds a row node; the table's own hash covers only its label and column definitions, so every other cell, and every value already taken from them, is untouched.
## 4. IDs that never move
After twelve corrections, the sentence about water is still `f929eq9`, the corrected cell is still `nkm3xn5`, and the sentence that lost its superscript is still `e4fkseg`.
The retired IDs are recorded in [`ids.jsonl`](../fixtures/packages/hale2026harmonic/ids.jsonl), so anything that pointed at them can follow the trail.
So is a legacy alias: `b-p01-text-003`, the ID an older tool gave the water sentence, still leads to `f929eq9`.
If a better converter comes along next year, its reading is aligned with this document sentence by sentence; every sentence it reads the same way keeps its ID, and every difference becomes a suspect rather than a silent change.
## 5. What happened to the notes
Before any correction, at revision 0, the team had already made seventeen anchors: notes, extracts and citations pointing into the first reading.
Here is how each resolves at revision 12 ([vectors/resolve.jsonl](../vectors/resolve.jsonl) records them all).
| Anchor on | Result | Why |
| --- | --- | --- |
| "H$_2$O" in the water sentence | `current` | Nothing in that sentence changed. |
| The cell `1.5260` | `current` | Untouched, though its table changed around it. |
| The water sentence, through the alias `b-p01-text-003` | `current` | The alias leads to the same node. |
| The cell that read `3l0.0` | `stale` | Its text changed and "3l0.0" is no longer in it; a person must look again, which is right, because the number changed. |
| "treated as zero" | `reattached` | The sentence changed twice, but the quote occurs in it exactly once, so the anchor is re-pinned. |
| "fitted constants" | `stale` | After the split, those words are in the new sentence `r1gacps`, not in `e4fkseg`. |
| "least squares", in the half-sentence `e9rh611` | `moved` | That ID was merged into `eajq93t`, which still holds the quote. |
| The paragraph `g44k023` | `moved` | Merged into `zyqr0yc`, which holds all of its sentences. |
| The running head | `stale` | Deleted, with no successor. |
| The equation | `reattached` | Only its number changed. |
| The figure | `reattached` | Only its label changed, by a `relabel`. |
| The CT–HC cell `1.090` in the garbled row | `reattached` | Its row was deleted, but exactly one cell in Table 1 has the same row and column headers and the same text: the new `e0n07n0`. |
| Section 2 | `changed-below`, drift high | The section itself is unchanged, but 53% of what is in it was touched. |
| Table 1 | `changed-below`, drift high | Rows were replaced and added. |
| The table body | `changed-below`, drift high | 75% of its cells were touched. |
| The paragraph whose broken sentence was rejoined | `changed-below`, drift high | Two of the three sentences it has held were touched. |
| The CT–CT row | `changed-below`, drift low | One of its five cells was corrected. |
A reader shows each note accordingly, and a Lode check, which is about exact words, accepts only `current`, `moved` and `reattached`.
## 6. Review and signatures
Now people check what matters.
The receipts are in [`receipts.jsonl`](../fixtures/packages/hale2026harmonic/receipts.jsonl), and the keys that signed them in [`roster.json`](../fixtures/packages/hale2026harmonic/roster.json).
- **Before the corrections**, `ana` had signed receipt `c5cx7n4tga6y9` for three cells at revision 0.
Two of them never changed, so the receipt still counts for them; for the corrected cell it no longer does, because its hash moved on.
- **After the corrections**, `ana` signs `eqgbbpwb6725v` at F3 for 23 nodes: the cells of Table 1's three data rows, its four column headers, the equation and the sentences around it.
`ben` spot-checks it with receipt `p38t0s91trb6j`: his signature over the receipt picks 10 nodes nobody could have chosen, and he finds no errors.
- `ana` signs `zaxky8c1y12tc` for two sentences of section 1, including the new `r1gacps`.
- `cara` signs `adtrpq41np7sd` for three sentences on page 2, then revokes it herself (`q0yg0w6ygwwxh`): she had compared them with the crops of another paper.
Those sentences fall back to their earlier levels, which is why `eajq93t` is still F0.
- `ana` signs `gena76nkb6cnc` for the figure's caption; `ben`'s spot-check `qab588b6xy94y` finds a missing period in `nmm4md0`, which suspends the receipt until she reviews the sitting again.
The document's report, in [`document.json`](../fixtures/packages/hale2026harmonic/document.json), counts the result: 28 leaves at F3, 12 at F2, 4 at F1 and 1 at F0.
## 7. Data out of the table
Table 1 becomes a dataset, `ds-hale-bond-params`, inside the package.
Its CSV keeps every digit as printed (`1.526` and `1.5260` stay different), and every value has an anchor to the cell that prints it, so a claim that rests on a value can show the cell.
All twelve value anchors resolve as `current`; the checker confirms each CSV value equals its cell's text.
## 8. Sharing without the text
The team wants to share its notes with a collaborator's machine but not the paper.
[`hale2026harmonic.checks-only`](../fixtures/packages/hale2026harmonic.checks-only/) is the same document with every leaf's text withheld and every hash replaced by an HMAC under the team's key.
IDs, labels, pages and boxes survive, so annotations still land in the right place; only someone holding the key and the text can check the values.
## 9. Your turn
[The examples](examples.md) show how to recompute one of these hashes with `sha256sum`, print the outline with a few lines of Python, and load the whole package into SQLite to query it.