Add paperspec correct, check math forms and datasets, and give the tools public names

The work the other paper tools planned for paperspec (paperdesk PLAN M5.4,
paperform PLAN 4.2):

- paperspec correct and paperspec.revision apply proposed corrections to a
  package, folder or zip, and write it only once it validates.
- doc.math.form is checked: label and style commands always, the form's own
  spelling when papermath is installed or forms= is passed.
- paperspec.datasets and seven doc.dataset.* rules check a package's datasets.
- fixtures/invalid gains suspect-witness and math-form-label (33 in all).
- Canonical JSON reads numbers as doubles, so 1e20 round-trips.
- The registries and the JSON-LD context ship in the wheel.
- Public names for what paperdesk and paperform imported as private ones,
  with the old names kept as aliases.
- The text and lossy-projection functions are about three times faster
  through the gate, tested equal to the old code on every code point.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013XwDhn4LkrKQfw4ArDEr2E
This commit is contained in:
alexmandClaude committed 2026-10-20 22:10:17 -04:00
1 parent 8c07dd7c15
commit 4fb8b4f5a7
70 files changed
+4470 -270

No files matched your search

+9 -7
View File
@@ -25,12 +25,12 @@ A conforming reader accepts each one and recomputes every hash it contains.
| Package | What it is for |
| --- | --- |
| `packages/hale2026harmonic/` | The main example, followed step by step in [the walkthrough](../docs/walkthrough.md): a chemistry paper that was read once with mistakes and then fixed by twelve corrections. It has a table with two levels of column headers and grouped rows, an equation inside a sentence, a footnote, a figure with text inside its panels and a real image, references, legacy aliases, boxes on fingerprinted pages, receipts in every state (counting, partly invalidated, spot-checked, revoked and suspended), the roster of public keys that checks their signatures, a dataset taken from the table, derived MathML, and the derived reading of three formulas in the math form `scienting-math/1`, one of which the form's tree cannot read ([spec/09 section 9.4](../spec/09-figures-tables-equations.md#94-the-math-form)). |
| `packages/hale2026harmonic.checks-only/` | The same document prepared for sharing within the team: text withheld, every hash an HMAC (a hash computed under a secret key) under the team key. The team key is derived from the team secret in `keys/` ([spec/05](../spec/05-hashes.md), [spec/12 section 12.5](../spec/12-package.md#125-text-policies)). It shows how a team can share its checks on a paywalled paper without sharing the paper. |
| `packages/okafor2025open/` | An open-access paper established from a JATS file (an XML format for journal articles) and a PDF that agree, so every leaf is at fidelity F2, corroborated by an independent witness. |
| `packages/smithlab2027charges/` | A short draft written in Typst (a typesetting language). One of its sentences holds a cross-reference whose target is the address `hale2026harmonic#a4jrz2t`, a sentence of the main example. |
| `packages/tiny2026example/` | A small document, the starting point of the correction vectors (`vectors/corrections.jsonl`) and of every broken package below. |
| `packages/tiny2026example.sct.zip` | The same document as a zipped package, written the way a producer makes a byte-identical archive: entries under one folder named by the key, sorted by path, stored, dated 1980-01-01, with no extra fields ([spec/12 section 12.2](../spec/12-package.md#122-paths)). A reader unpacks it to exactly the folder beside it. |
| `packages/hale2026harmonic-damb/` | The main example, followed step by step in [the walkthrough](../docs/walkthrough.md): a chemistry paper that was read once with mistakes and then fixed by twelve corrections. It has a table with two levels of column headers and grouped rows, an equation inside a sentence, a footnote, a figure with text inside its panels and a real image, references, legacy aliases, boxes on fingerprinted pages, receipts in every state (counting, partly invalidated, spot-checked, revoked and suspended), the roster of public keys that checks their signatures, a dataset taken from the table, derived MathML, and the derived reading of three formulas in the math form `scienting-math/1`, one of which the form's tree cannot read ([spec/09 section 9.4](../spec/09-figures-tables-equations.md#94-the-math-form)). |
| `packages/hale2026harmonic-damb.checks-only/` | The same document prepared for sharing within the team: text withheld, every hash an HMAC (a hash computed under a secret key) under the team key. The team key is derived from the team secret in `keys/` ([spec/05](../spec/05-hashes.md), [spec/12 section 12.5](../spec/12-package.md#125-text-policies)). It shows how a team can share its checks on a paywalled paper without sharing the paper. |
| `packages/okafor2025solvent-acth/` | An open-access paper established from a JATS file (an XML format for journal articles) and a PDF that agree, so every leaf is at fidelity F2, corroborated by an independent witness. |
| `packages/writer2027fixed-ynr7.v3/` | A short draft written in Typst (a typesetting language). One of its sentences holds a cross-reference whose target is the address `hale2026harmonic-damb#a4jrz2t`, a sentence of the main example. |
| `packages/tiny2026tiny-j4qm/` | A small document, the starting point of the correction vectors (`vectors/corrections.jsonl`) and of every broken package below. |
| `packages/tiny2026tiny-j4qm.sct.zip` | The same document as a zipped package, written the way a producer makes a byte-identical archive: entries under one folder named by the key, sorted by path, stored, dated 1980-01-01, with no extra fields ([spec/12 section 12.2](../spec/12-package.md#122-paths)). A reader unpacks it to exactly the folder beside it. |
## Records
@@ -64,7 +64,7 @@ Each record is a single file, or a folder where a format needs several files.
## Broken packages
Each folder in `invalid/` holds a `package/` folder, or a `package.sct.zip` archive, that breaks exactly one rule, built from `packages/tiny2026example/` by one edit, and an `expect.json` that names the rule.
Each folder in `invalid/` holds a `package/` folder, or a `package.sct.zip` archive, that breaks exactly one rule, built from `packages/tiny2026tiny-j4qm/` by one edit, and an `expect.json` that names the rule.
A conforming implementation refuses every one of them and should report that rule.
`registry/rules.json` lists every rule ID with its meaning, and a validator reports each failure as `path:line: rule: message` ([spec/22 section 22.2](../spec/22-conformance.md#222-validating-a-package)).
@@ -85,6 +85,7 @@ A conforming implementation refuses every one of them and should report that rul
| `location-unknown-page/` | A location on a page no witness has. | `doc.location.page` |
| `manifest-hash/` | A file changed after the manifest was written. | `package.manifest.hash` |
| `manifest-key/` | The manifest names another key than document.json. | `package.manifest.key` |
| `math-form-label/` | A `scienting-math/1` source that still holds its `\tag`, which the form lifts out. | `doc.math.form` |
| `missing-required-file/` | A required file is missing. | `package.file.required` |
| `not-canonical/` | A line that is valid JSON but not canonical JSON (the fixed form of RFC 8785, which gives equal data identical bytes). | `package.file.canonical` |
| `receipt-root/` | A receipt whose Merkle root (one hash that summarizes a list of hashes) does not recompute. | `doc.receipt.root` |
@@ -95,6 +96,7 @@ A conforming implementation refuses every one of them and should report that rul
| `root-mismatch/` | document.json's root is not the root node's node_hash. | `doc.hash.root` |
| `signer-mismatch/` | A receipt whose signature was made with another key than the signer it names. | `doc.receipt.signature` |
| `style-sup-sub/` | A run cannot be both a subscript and a superscript. | `doc.run.style` |
| `suspect-witness/` | A suspect whose second reading names a witness the package does not have. | `doc.suspect.witness` |
| `text-edited/` | Text changed without its hashes. | `doc.hash.text-hash` |
| `unknown-kind/` | Kinds outside the list are refused. | `package.schema` |
| `unlisted-file/` | A file the manifest does not list. | `package.manifest.unlisted` |