alexmandClaude
ea0a7efa78
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):
- Zipped packages: the oracle writes and reads `.sct.zip` archives under
the rules of spec/12 §12.2; validate.py and load.py accept them; a
byte-identical zip of the tiny package joins the fixtures, and two
broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
oracle and pinned by vectors/targets.jsonl under each digit rule; the
rule now covers cells as well as sentences and the leaves of anchored
containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
accessible export can find a confirmed figure description; the layer
fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
used them.
Audit fixes, each with a vector or a broken package where one was
missing:
- Folder packages: symbolic links are refused on validate and load,
manifest paths must not collide under case folding, listed files are
matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
a lone surrogate or invalid UTF-8 is refused instead of translated;
integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
signed; a qa receipt's nodes must be the sample at the original's
hashes, its scope must name qa.of, its errors must be sampled nodes;
statuses are computed once, iteratively, so chains of spot-checks
hundreds deep settle; Base64 and manifest shape errors become refusals
instead of tracebacks; counts and the fidelity of roster-less packages
are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
validated against the node schema and the references between nodes
are checked, so a set to a bad value, an insert with unknown
properties or a delete of a referenced header cell is refused; a
relabel to null drops the label; a reclassify drops foreign
attributes; a split strips whitespace across runs; ids.jsonl order
follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
"10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
revocation may be carried by several packages.
Checking:
- tools/stress.py: seeded randomized property tests over canonical JSON,
normalization, notation, random correction walks, anchor resolution,
Merkle roots, spot-check samples, error bounds, fingerprints, mutated
archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
the repository, the zipped fixture and crafted archives, the checklist
coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
151 lines
12 KiB
Markdown
151 lines
12 KiB
Markdown
# 8. Fidelity, receipts and spot-checks
|
||
|
||
## 8.1 Levels of fidelity
|
||
|
||
| Level | Meaning |
|
||
| --- | --- |
|
||
| F0 converted | One reading, not corroborated. |
|
||
| F1 resolved | Every suspect in the node was answered against the page image and passed the gate. |
|
||
| F2 corroborated | An independent witness agrees, by `match_hash` and complex keys, or the text is born digital, as a Typst or LaTeX source is. |
|
||
| F3 proofread | A named person compared the node with the page image, and a valid receipt they signed lists the node at its current `node_hash`. |
|
||
|
||
Levels apply to leaves.
|
||
The pipeline records F0, F1 or F2 in a leaf's `fidelity`; F3 comes only from receipts.
|
||
A correction that changes a leaf drops it back: the gate's crop evidence makes it F1 at most, until it is corroborated or proofread again.
|
||
|
||
## 8.2 Levels
|
||
|
||
A leaf's level is the highest of its recorded `fidelity` and the levels of every valid receipt ([8.6](#86-receipt-status)) that lists it at its current `node_hash`.
|
||
A receipt at a lower level never lowers a node; only a revocation, a failed spot-check or a correction does.
|
||
A spot-check's `qa:` receipt ([8.8](#88-spot-checks)) raises the nodes it sampled, except those its `errors` name.
|
||
A document reports, in `document.json`, the lowest level among its leaves and the count at each level.
|
||
|
||
## 8.3 Receipts
|
||
|
||
A paper has thousands of nodes, and nobody signs thousands of times, so people sign receipts: one signature over a Merkle root of everything reviewed in a sitting.
|
||
|
||
| Property | Required | Holds |
|
||
| --- | --- | --- |
|
||
| `format` | Yes | `scienting-receipt/1`. |
|
||
| `id` | Yes | The record ID ([4.6](04-identity.md#46-record-ids)). |
|
||
| `doc` | Yes | The document key. |
|
||
| `rev` | Yes | The revision reviewed. |
|
||
| `level` | Yes | `F1`, `F2` or `F3`. |
|
||
| `scope` | Yes | `all`, `section:<id>`, `page:<n>`, `targeted:<name>` (such as `targeted:lode`), or `qa:<receipt id>` for a spot-check. |
|
||
| `nodes` | Yes | `[[id, node_hash], …]`, sorted by ID without repeats. |
|
||
| `records` | No | `[[kind, id, hash], …]` for other things signed in the sitting: `annotation`, `digitization`, `correction`, `layer` or `checklist`. |
|
||
| `root` | Yes | The Merkle root ([8.5](#85-merkle-roots)). |
|
||
| `evidence` | Yes | What the reviewer looked at: `{kind, sha256, node?, page_fp?, box?, note?}` with `kind` `crop`, `witness` or `source-file`, or `{kind: "person", by}`. A regenerated crop is different evidence. |
|
||
| `by` | Yes | The signer's roster key. |
|
||
| `at` | Yes | When it was signed. |
|
||
| `qa` | For `qa:` scopes | The spot-check ([8.8](#88-spot-checks)). |
|
||
| `export_of` | No | The receipt this one re-issues under a keyed scheme. |
|
||
| `keyed` | No | The scheme its values are keyed under, when it leaves the team. |
|
||
| `signature` | Yes | `{scheme, key, value}` ([8.4](#84-record-hashes-and-signatures)). |
|
||
|
||
A receipt lists nodes, records, or both.
|
||
A correction invalidates a receipt for the corrected node only, because the node's `node_hash` no longer matches; the rest still counts.
|
||
|
||
A receipt shared outside the team cannot carry plain node hashes, which would leak short texts, and its signature covers them.
|
||
So an export re-issues it: a new receipt with `export_of`, values under the export's key, signed by the original reviewer or a team key with the `export` purpose.
|
||
|
||
## 8.4 Record hashes and signatures
|
||
|
||
A signed record's record hash is SHA-256 of the canonical JSON of the record without `id` and `signature`.
|
||
Its `id` is the 13-character letter-first encoding of that hash.
|
||
Its signing payload is the canonical JSON of `{"format": <the record's format>, "hash": <the record hash>}`.
|
||
|
||
| Scheme | `value` |
|
||
| --- | --- |
|
||
| `ed25519` | The RFC 8032 signature of the payload, in standard Base64 with padding. |
|
||
| `ssh` | An SSH signature of the payload in the namespace `scienting`, armored. |
|
||
| `gpg` | A detached, armored OpenPGP signature of the payload. |
|
||
| `sigstore` | A Sigstore bundle for the payload, as JSON text. |
|
||
|
||
`signature.key` is the signer's roster key and equals `by`.
|
||
Self-hosted deployments sign with team keys or a private Sigstore deployment; the public Sigstore instance, whose log permanently records each signer's identity, is for public layers only.
|
||
`signatures.jsonl` verifies one receipt and the RFC 8032 test vector.
|
||
|
||
## 8.5 Merkle roots
|
||
|
||
A receipt's items are `["node", id, node_hash]` for each listed node, in order, followed by its `records`.
|
||
The root follows RFC 6962: a leaf hash is SHA-256 of the byte 0x00 and the item's canonical JSON; an interior hash is SHA-256 of 0x01 and the two child hashes; a list of *n* > 1 items splits after the largest power of two less than *n*.
|
||
RFC 6962 has since been replaced by RFC 9162, which defines the same tree.
|
||
A root over one item is that item's leaf hash; a receipt with no items is invalid.
|
||
The tree allows a proof that one node was covered without revealing the others; `merkle.jsonl` gives roots for one to seven items.
|
||
|
||
## 8.6 Receipt status
|
||
|
||
A receipt is:
|
||
|
||
1. **invalid** when its root, ID or signature does not verify, or its signer's key was not valid when it was signed (`registered_at` ≤ `at` < `revoked_at`);
|
||
2. otherwise **revoked** when a valid revocation names it, or names its signer with a period that contains its `at`;
|
||
3. otherwise **suspended** when a valid spot-check of it found an error;
|
||
4. otherwise **valid**.
|
||
|
||
Only valid receipts set levels.
|
||
A suspended receipt stays suspended; its reviewer reviews the sitting again and signs a new receipt, which needs its own spot-check.
|
||
A spot-check is itself a receipt, so its status may depend on its own spot-check; statuses are settled from the end of each chain, and the receipts of a chain that loops back on itself are invalid.
|
||
|
||
## 8.7 Revocations and the roster
|
||
|
||
A revocation, `scienting-revocation/1`, names one `receipt`, or a `signer` and a `period` `{from, to}`, with a `reason`, and is signed like a receipt.
|
||
It is valid when its signature verifies and its signer is the revoked receipt's signer, the signer it names, or a key whose `role` in the roster is `owner`.
|
||
|
||
The roster, `scienting-roster/1`, lists the keys that may sign: each with an `id`, a `person`, a `role`, a `scheme`, its `public` key (Base64 for Ed25519), its `purposes` (`sign`, `spot-check`, `export`), `registered_at` and `revoked_at`.
|
||
The `role` is the holder's role in the team that keeps the roster, `owner`, `editor`, `reader` or `guest`, as in a project ([20](20-projects.md)).
|
||
Because the roster carries it, a package that carries its roster carries everything needed to judge its revocations.
|
||
A package MAY carry the roster its receipts need as `roster.json`, so its receipts verify without the store.
|
||
The roster is meant to be the one a store verifies against; see the [rationale](../docs/rationale.md#known-limits).
|
||
|
||
## 8.8 Spot-checks
|
||
|
||
For each F3 receipt, a second reviewer checks a sample of its nodes and signs a `qa:` receipt.
|
||
A seed the reviewer controls could be ground, by varying the receipt until the sample falls on careful work, so the seed is a signature nobody can steer:
|
||
|
||
1. The seed is the checker's Ed25519 signature over the canonical JSON of `{"format": "scienting-spot-check/1", "receipt": <the original's record hash>}`, made with a key registered in the roster with the `spot-check` purpose before the original was signed.
|
||
Ed25519 signatures are deterministic, so the checker cannot choose it, and the reviewer cannot predict it.
|
||
2. The sample size is the smaller of *n* and the larger of `minimum` and ⌈`rate` × *n*⌉, for the original's *n* nodes; the defaults are a rate of `0.05` and a minimum of 10.
|
||
3. Rank the original's node IDs by the hexadecimal SHA-256 of the seed's 64 bytes followed by the ID's UTF-8, then by ID, and take the first ones.
|
||
|
||
The `qa` object records `of`, `key`, `seed` (Base64), `rate`, `minimum`, the `sample` sorted by ID, and the `errors` found, each `{node, kind?, note?}`.
|
||
The `qa:` receipt's `nodes` are the sampled nodes at the hashes the original lists, its `scope` is `qa:` and the original's ID, every error names a sampled node, and it is signed at or after the original.
|
||
Anyone can recompute the sample; `spotcheck.jsonl` does.
|
||
The payload's identifier, `scienting-spot-check/1`, is registered with the other formats so that no file ever carries it by mistake; it names a signing payload, never a file.
|
||
|
||
## 8.9 Reviewer error bounds
|
||
|
||
One sample proves little; what someone relying on the work needs is a bound on each reviewer's error rate across all their samples.
|
||
With *k* errors in *n* sampled nodes, the bound is the one-sided Clopper–Pearson upper confidence bound at confidence *c*, by default 0.95: the rate *p* at which the probability of at most *k* errors in *n* is 1 − *c*, or 1 when *k* = *n*.
|
||
It is reported to six decimal places, rounded up.
|
||
With no errors the bound is about 3/*n*, so zero errors in 300 sampled nodes bounds a rate below 1%; `bounds.jsonl` gives cases.
|
||
|
||
A QA report, `scienting-qa-report/1`, states for each reviewer who signed a delivery the nodes sampled, the errors found, the confidence, the method `clopper-pearson` and the bound.
|
||
It is the record a buyer keeps, so it is signed like a receipt ([8.4](#84-record-hashes-and-signatures)): `by` names the roster key of the team that made the delivery, `id` derives from the record hash, and `signature.key` equals `by`.
|
||
|
||
## 8.10 Targeted F3
|
||
|
||
Most of a paper is never relied on, so a team proofreads the target set: the nodes its claims, data and quotations rest on.
|
||
|
||
1. The anchored nodes: every live node that an extract, a dataset value, a confirmed `claim-link`, a draft quotation or a pinned citation anchors, each anchor resolved first ([6.5](06-anchors.md#65-resolution)).
|
||
An anchored leaf is a target; an anchored container contributes its leaves.
|
||
2. Every `equation` and `equation-line`.
|
||
3. Under the project's digit rule ([20](20-projects.md)): with `all`, every `sentence` and `cell` with a digit in a `text` or `math` run; with `anchored-sections`, only those in a section that holds an anchored node; with `none`, none.
|
||
Digits are characters of General Category Nd.
|
||
Digits in `cite`, `xref` and `fnmark` runs do not count, so citation marks, "Figure 3" and footnote marks pull nothing in.
|
||
A node's section is its nearest ancestor-or-self of kind `section`, or the `front` or `back` matter when it has none.
|
||
|
||
The set is reported in document order, each node with the first reason that applies: `anchored`, `equation` or `digits`.
|
||
Neighbors shown for context are not in the set, and nothing new comes to rest on a node below F3 without the node joining the set.
|
||
`targets.jsonl` gives the set of `fixtures/packages/hale2026harmonic` under each digit rule.
|
||
|
||
## 8.11 Checklists
|
||
|
||
An export of a document, such as accessible HTML, can be checked against a standard, and that check is a record a buyer keeps.
|
||
A checklist, `scienting-checklist/1`, holds the `doc` and `rev` checked; the `standard`, `{name, level?, url?}`, such as WCAG 2.1 at level AA; the `export` checked, `{kind, sha256, size?}`, so the record binds to one file; its `items`, each a `criterion`, its `result` (`pass`, `fail` or `not-applicable`), the `method` (`manual`, or `automated` with the `tool` and its version) and an optional `note`; and `by` and `at`.
|
||
Its content hash is SHA-256 of the canonical JSON of the record without `id`, and its `id` is the record ID of that hash ([4.6](04-identity.md#46-record-ids)).
|
||
A checklist carries no signature of its own: the person who checked signs a receipt that lists it as `["checklist", id, hash]` ([8.3](#83-receipts)), so one sitting covers a checklist together with the nodes read for it.
|
||
The receipt covers the checklist only at that hash, so a checklist changed afterwards is uncovered.
|
||
`fixtures/records/checklist/` holds a checklist for an accessible HTML export of the example paper and the receipt that covers it.
|
||
The format is `provisional` ([0.4](00-introduction.md#04-status)).
|