15 KiB
Concepts
Each section introduces one idea, shows it in the example paper hale2026harmonic (a synthetic chemistry paper in fixtures/packages/), and links to the chapter that defines it exactly.
Works, versions, witnesses and documents
Four things are easy to confuse, so the formats keep them apart.
| Object | In the example |
|---|---|
| A work is the paper across all its versions. | Hale and Osei's paper on harmonic bond parameters, doi:10.5555/example.hale.2026. |
| A version is one expression of it: a preprint, the version of record, an erratum, the Supporting Information. | The version of record. |
| A witness is one file of a version, identified by its SHA-256. | The publisher's PDF, witness qptsanv. |
| A document is the text established for one version, from one or more witnesses. | hale2026harmonic. |
A document's key is its work's key, with a suffix for anything but the version of record: hale2026harmonic.si would be its Supporting Information.
See chapter 2.
A document is a tree of nodes
Every part of the text is a node: the document, its sections, headings, paragraphs and sentences, figures and their panels and captions, tables with their rows and cells, equations, footnotes and references. The example has 75 nodes; its outline is:
bzqvbhd 1 Introduction
qn08tj9 equation (1)
f0tj6ca 2 Methods
ajfvq4s figure Figure 1
gm5z4rx table Table 1
nodes.jsonl holds one node per line, in reading order, each with its kind, parent, position, content and hashes.
A sentence's content is a list of runs: plain text, styled text, inline math, citations, cross-references, footnote marks, links, and display equations the sentence runs through.
This is one table cell, shortened:
{"id":"nkm3xn5","kind":"cell","parent":"zhe4g0k","ord":1,
"attrs":{"column":"…","role":"data","rowspan":1,"colspan":1},
"content":[{"kind":"text","text":"310.0"}],
"fidelity":"F1","text_hash":"…","match_hash":"…","align_key":"…","own_hash":"…","node_hash":"21f13b…"}
See chapter 3.
IDs are names, not positions
nkm3xn5 says nothing about where the cell is.
That is deliberate: an ID that encoded a page or an ordinal would change when a better converter ordered blocks differently.
IDs are minted once from the document key, the node's kind, its text and an occurrence count, so a fresh conversion of the same input gives the same IDs, and then they never change.
When two nodes merge, one ID is retired and points at its successor; when an artifact is deleted, its ID is retired with no successor.
Legacy IDs, such as the block IDs older tools used, live on as aliases: in the example, b-p01-text-003 still leads to sentence f929eq9.
See chapter 4.
Addresses
An address names a node, and optionally pins it to a state and narrows it to a span:
| Address | Names |
|---|---|
hale2026harmonic#f929eq9 |
The sentence about water, whatever it says now. |
hale2026harmonic#f929eq9@cd6cdb |
That sentence as it read when its node_hash began cd6cdb. |
hale2026harmonic#f929eq9@cd6cdb[29:44] |
Code points 29 to 44 of it in that state: "first test case". |
smithlab/hale2026harmonic#f929eq9 |
The same sentence in the smithlab store. |
On the web the ID is the fragment, so a link to …/smithlab/hale2026harmonic#f929eq9 lands on the sentence.
See chapter 6.
Five hashes, five jobs
Every node carries five content values.
| Value | Covers | Answers |
|---|---|---|
text_hash |
The text exactly, normalized. | Is this the same quotation, character for character? |
match_hash |
The text with notation folded to one form. | Do two witnesses really agree? |
align_key |
The text folded loosely. | Is this probably the same sentence? |
own_hash |
The node's own content, without its children. | Has this section itself changed, as opposed to something inside it? |
node_hash |
The node and everything below it. | Has anything here changed at all? |
The middle two exist because one hash cannot do both of their jobs.
Chemistry writes the same thing many ways, and every one of these sentences has the same match_hash:
| As stored | Strict projection |
|---|---|
The solvent is $H_2O$. |
The solvent is H_{2}O. |
The solvent is H$_2$O. |
The solvent is H_{2}O. |
The solvent is H and a subscript 2, then O. |
The solvent is H_{2}O. |
The solvent is H₂O. |
The solvent is H_{2}O. |
The solvent is $H_2 O$. in Typst |
The solvent is H_{2}O. |
But OCR often drops a superscript, and "103" must never be certified as agreeing with 10³.
So "A rate of 103 per second." and "A rate of 10³ per second." share an align_key, which proposes that they are the same sentence, while their match_hash values differ, which raises a suspect of kind number for a person to resolve.
The same care keeps "The energy" followed by citation 3 from matching the text "The energy3".
See chapter 5 and vectors/projections.jsonl.
Anchors
An anchor is how anything points into a document: a Lode extract, an annotation, a dataset value, a draft's citation. It is a pinned address with enough context to find its text again after the document changes:
{"format":"scienting-anchor/1","store":"smithlab","doc":"hale2026harmonic","node":"f929eq9","kind":"sentence",
"own_hash":"d5b2eb…","node_hash":"cd6cdb…","span":[29,44],
"quote":{"exact":"first test case","prefix":"Water, H$_2$O, served as the ","suffix":", with a bending band near 1595 "},
"rev":12}
rev is the document's revision when the anchor was made, counted in corrections rather than clock time.
When the document changes, an anchor resolves one of five ways:
| Result | Means | In the example |
|---|---|---|
current |
Nothing it points at has changed. | A note on the water sentence. |
changed-below |
A container is unchanged in itself, but something inside it changed; a drift figure says how much. | A note on Table 1 after a row was inserted. |
moved |
Its node was merged into another that still holds the quote. | A note on half of a sentence that was wrongly split, then rejoined. |
reattached |
Its node changed, but the quote is still there exactly once, so it is re-pinned. | A note on "treated as zero" in a sentence whose superscript was fixed. |
stale |
It can no longer be trusted; a person must re-anchor it. | A note on the cell that read "3l0.0" before it was corrected to "310.0". |
See chapter 6; the walkthrough shows all five.
Corrections
A document is never edited in place.
Every fix is a correction: an operation with its evidence, its author and a reason, appended to corrections.jsonl.
{"format":"scienting-correction/1","index":1,"op":"replace","target":"nkm3xn5",
"find":"3l0.0","replace":"310.0",
"evidence":[{"kind":"crop","sha256":"9b6fe6…"}],"author":{"key":"ana"},
"reason":"OCR read the digit 1 as the letter l.","at":"2026-10-02T10:00:00Z",
"before":"2fd1e9…","after":"21f13b…","touched":["nkm3xn5"]}
The operations are replace, insert, delete, split, merge, move, reclassify, relabel and set.
A gate refuses any correction that would change a digit without a crop of the page as evidence, or that would let a digit change anywhere it did not declare.
See chapter 11.
Fidelity
Every leaf has a level that says how far its text was checked.
| Level | Means |
|---|---|
| F0 converted | One reading, not corroborated. |
| F1 resolved | Every doubt about it was answered against the page image. |
| F2 corroborated | An independent witness agrees exactly, or the text was born digital. |
| F3 proofread | A named person compared it with the page and signed for its current text. |
A level counts only at the node's current hash, so a correction drops a proofread sentence back until someone signs for it again. The example reports 28 leaves at F3, 12 at F2, 4 at F1 and 1 at F0. See chapter 8.
Receipts and spot-checks
Nobody signs three thousand nodes one at a time.
A reviewer signs a receipt: one signature over a Merkle root of everything reviewed in a sitting, with the evidence looked at.
In the example, the reviewer ana signs one receipt for 23 nodes: Table 1's cells, the equation and the sentences around it.
Signing once per sitting invites rubber-stamping, so a second reviewer spot-checks a sample. The sample is chosen by the checker's Ed25519 signature over the receipt, which neither reviewer can steer, and anyone can recompute it. Across samples, the record states a bound on each reviewer's error rate: zero errors in 10 sampled nodes bounds the rate below 26%, and zero errors in 300 bounds it below 1%.
A spot-check that finds an error suspends the receipt; a revocation withdraws it. Either way the nodes fall back to the level they had without it. See chapter 8.
Page fingerprints
Publishers stamp each download with the institution and time, so two teammates' copies of one paper rarely share a SHA-256.
Each page therefore also has a fingerprint: a hash of its size, its text with the stamp lines removed, and its images.
Two downloads of the same page stamped "Downloaded via UNIVERSITY OF EXAMPLE on October 11, 2026" and "Downloaded via EXAMPLE PHARMA INC on March 2, 2027" share the fingerprint 6272e6f3…; change one digit of the text and it becomes b545d285….
Geometry and digitizations point at fingerprints, so a re-download keeps every box and every value read off a plot.
See chapter 7.
Keeping text private
A team may annotate a paper it may not share. A checks-only package keeps every ID, label, page and box, but withholds the text, and keys every hash, because a plain hash of a short text such as "310.0" is recovered by hashing guesses. Inside the team, values are plain SHA-256; exported to another of the team's machines, they are HMACs under the team's key; in a public layer, the key is derived from the paper's page fingerprints, so only someone holding the paper can check them. fixtures/packages/hale2026harmonic.checks-only is the example paper exported this way. See chapter 5.8.
Rights
What a team may do with a paper depends on how it holds the paper. Every witness carries a rights record: the license as stated, the basis (an open license, a subscription, a collective license, the team's own work), and which uses that allows, including AI uses by purpose and by where the processing runs. Every processing stage declares what it needs and runs only when the rights allow it.
The example PDF is held on a subscription that allows text mining and internal AI use on the team's own machines.
So conversion by a local model is allowed, conversion by an outside service is refused for lacking ai/internal/outside, and model review in the team's cloud is refused for lacking ai/internal/tenancy? (the question mark means unknown) until a project owner signs a policy that settles it.
See chapter 14.
Layers and annotations
Everything people and models add is an annotation, grouped in layers such as "Novice glosses" or "Journal club, March". Annotations are W3C Web Annotations whose selector is an anchor, so any Web Annotation tool can read them. This gloss, shortened, explains a phrase for a first-year student:
{"type":"Annotation","motivation":"describing","status":"confirmed",
"body":{"type":"TextualBody","kind":"gloss","audience":"novice",
"value":"A *harmonic* term treats a bond like a spring: the energy grows with the square of the stretch."},
"target":{"type":"SpecificResource","source":"urn:scienting:doc:smithlab/hale2026harmonic",
"selector":[{"type":"AnchorSelector","anchor":{"node":"sxgcc9h","span":[111,130],"…":"…"}},
{"type":"TextQuoteSelector","exact":"bond force constant"}]}}
One rule decides what is a claim and what is a layer: a statement true independently of the paper belongs in Lode as a checked claim; a statement about reading the paper belongs in a layer.
Model-written text is always a generated annotation, never source.
See chapter 18.
Exercises
Because every sentence and cell has an address, the answer to "find the evidence" is an address, and an address can be graded.
An exercise asks a student to find the cell giving the Set A bond constant for CT–CT; a student who points at nkm3xn5 scores 1 of 1 with "That is the cell.", and one who points at the Set B cell scores 0 with "Look in Table 1, row CT–CT."
Students submit under pseudonyms, and a key whose anchors have gone stale is never used to grade.
See chapter 19.
Datasets, digitizations and views
Numbers live in prose, in tables and inside figures; all three should end up as data a team can plot and cite. A dataset is a Frictionless Data Package with units, uncertainty, and an origin for every value. The example's Table 1 becomes a CSV whose every value is anchored to the cell that prints it, digits exactly as printed:
key,bond,set,k_r,r_eq
ct-ct-a,CT–CT,A,310.0,1.526
ct-ct-b,CT–CT,B,305.2,1.5260
A digitization records how values were read off a plot (axes, calibration, series, uncertainty) and counts only once a person confirms it against an overlay. A view is a Vega-Lite specification that names its data by dataset ID and hash. See chapters 15, 16 and 17.
Packages, databases and bundles
A document travels as a package: a folder of canonical JSON files.
hale2026harmonic/
MANIFEST.json every other file, with its SHA-256; its own hash pins the package
document.json the work, version, metadata, policy, root hash and fidelity
nodes.jsonl one node per line, in reading order
witnesses.json each witness and its rights
pages.jsonl each page and its fingerprint
locations.jsonl where each node is on each page
references.jsonl parsed references
corrections.jsonl every correction, in order
receipts.jsonl receipts and revocations
ids.jsonl retired IDs and legacy aliases
roster.json the keys that signed
provenance.json tools, versions and where each stage ran
assets/ images, by SHA-256
datasets/ datasets reported from the document
derived/ MathML and other renderings, rebuildable
Every line is canonical JSON, so a corrected sentence is a one-line diff, and every file has exactly one valid byte sequence. The same data loads into tables that run on SQLite, Dolt and PostgreSQL, and unloads back to the same bytes. A bundle is one SQLite file with every table and a full-text index, which a reader or an MCP server can query offline. See chapters 12 and 13.