feat: minted keys, VRF seeds, RFC 9162 proofs, history roots, tag URIs and a transparency log

Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
  Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
  where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
  openalex in same_as, else the folded title, year and family name. A
  collision lengthens the later key by two characters; a key is minted
  once and never recomputed. A document key adds .si, .v<seq> or .<type>
  for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
  okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
  every node and record ID was re-minted and the docs rewritten to match.
  The validator checks the document key against the work key and version
  (doc.key.work) and the hash part against the identifier (doc.key.suffix).
  vectors/keys.jsonl pins the cases paperstack's tests pin.

Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
  implemented in tools/oracle/vrf.py in plain Python and reproduced
  against the RFC's Example 16; the sample ranks by the VRF output. The
  earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
  verify_inclusion, consistency_proof, verify_consistency) with
  scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
  the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
  checkpoints with key hashes as transparency logs share them, the rule
  that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).

Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
  namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
  the checker refuses another version.

Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
  checkpoint records; a check that every ID and hash the prose quotes
  exists in the fixtures or vectors; three more broken packages
  (history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
  keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
  section, media-type registration and OCI distribution notes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-14 23:21:57 -04:00
1 parent 76d76353d9
commit a09d158e16
307 files changed
+2698 -1497

No files matched your search

+9 -9
View File
@@ -19,11 +19,11 @@ The identifiers appear inside hashed data and test vectors, so they are not rena
Suppose a research team keeps fifty papers it needs to understand completely.
A page number names a region of a PDF and cannot name a statement, and it changes between a preprint and the published version.
The team wants to point at one sentence or one table cell, such as `hale2026harmonic#sxgcc9h@76e06b[111:130]` in the fixtures, and have that pointer keep working after someone fixes a misread digit elsewhere in the paper.
The team wants to point at one sentence or one table cell, such as `hale2026harmonic-damb#rr20tev@76e06b[111:130]` in the fixtures, and have that pointer keep working after someone fixes a misread digit elsewhere in the paper.
The pointer has four parts:
- The document key, `hale2026harmonic`.
- After `#`, the node ID, `sxgcc9h`.
- The document key, `hale2026harmonic-damb`.
- After `#`, the node ID, `rr20tev`.
- After `@`, the first six digits of the node's hash, `76e06b`, so the pointer names the sentence as it then read.
- In brackets, the code-point range of the words inside the sentence, `[111:130]`.
It also wants to know who compared a passage with the page image, and how far.
@@ -67,7 +67,7 @@ The design choices that matter for comparison:
Resolving it gives one of five results: `current`, `changed-below`, `moved`, `reattached` or `stale`, with a drift measure for `changed-below` ([spec/06](../spec/06-anchors.md)).
- **Corrections:** Corrections are appended, never edited.
Each carries evidence, an author and a reason, and a gate refuses a digit change without image evidence ([spec/11](../spec/11-corrections.md)).
In `fixtures/packages/hale2026harmonic`, correction 1 replaces `3l0.0` with `310.0` because "OCR (optical character recognition) read the digit 1 as the letter l", with a crop of the page as evidence.
In `fixtures/packages/hale2026harmonic-damb`, correction 1 replaces `3l0.0` with `310.0` because "OCR (optical character recognition) read the digit 1 as the letter l", with a crop of the page as evidence.
- **Fidelity and receipts:** Levels F0 (converted) to F3 (proofread).
F3 comes only from a signed receipt that lists the node at its current `node_hash`; a correction drops the node back ([spec/08](../spec/08-fidelity.md)).
- **Scale:** The formats are built for a team of roughly five to 20 people with a few dozen to a few hundred papers.
@@ -135,7 +135,7 @@ paperspec is not a publisher interchange format and does not replace JATS ([spec
*Interoperation.*
JATS is an input.
`jats` is a witness kind, and the fixture `okafor2025open` was established from a JATS file and a PDF that agree ([fixtures/README.md](../fixtures/README.md)).
`jats` is a witness kind, and the fixture `okafor2025solvent-acth` was established from a JATS file and a PDF that agree ([fixtures/README.md](../fixtures/README.md)).
The document's `meta` is one CSL-JSON item, not JATS front matter ([spec/02 §2.2](../spec/02-model.md#22-documentjson)).
The specification defines no JATS exporter.
@@ -427,8 +427,8 @@ Keep using it as a reading and distribution format.
*Status.*
The specification is version 0.1.0.
`registry/formats.json` lists 29 formats: 18 are `candidate` (complete, with vectors and fixtures, ready to implement) and 11 are `provisional` (complete; expected to change as implementations gain experience): project, bundle, dataset, digitization, view, snapshot, draft-ids, exercise, submission, grade and checklist ([spec/00 §0.4](../spec/00-introduction.md#04-status)).
The repository holds 23 chapters, 39 JSON Schemas, 21 vector files, five valid fixture packages, one of them also zipped, and 28 invalid ones, each naming the rule it breaks, and a reference checker, `tools/check.py`.
`registry/formats.json` lists 32 formats: 19 are `candidate` (complete, with vectors and fixtures, ready to implement) and 13 are `provisional` (complete; expected to change as implementations gain experience): project, bundle, dataset, digitization, view, snapshot, draft-ids, exercise, submission, grade, checklist, log-entry and checkpoint ([spec/00 §0.4](../spec/00-introduction.md#04-status)).
The repository holds 24 chapters, 42 JSON Schemas, 25 vector files, five valid fixture packages, one of them also zipped, and 31 invalid ones, each naming the rule it breaks, and a reference checker, `tools/check.py`.
Licenses: prose CC BY 4.0; schemas, SQL, the JSON-LD context, registries, vectors and fixtures CC0 1.0; tools Apache-2.0 ([README.md](../README.md#licenses)).
*No adoption claim.*
@@ -466,12 +466,12 @@ These are the places where the guides and the specification differ, and this pag
4. **Five hashes everywhere:** `docs/overview.md` lists the things that get "a permanent ID and five content hashes" without naming them.
Table columns and figure-text items have IDs, are not nodes, and carry no five hashes ([spec/04 §4.2](../spec/04-identity.md#42-minting-node-ids), [spec/09 §9.1](../spec/09-figures-tables-equations.md#91-figures-and-panels)).
5. **Dataset digits:** spec/15 says values keep their printed digits, and in JSON they are strings.
The fixture descriptor `fixtures/packages/hale2026harmonic/datasets/ds-hale-bond-params/datapackage.json` types the field `k_r` as `number`.
The fixture descriptor `fixtures/packages/hale2026harmonic-damb/datasets/ds-hale-bond-params/datapackage.json` types the field `k_r` as `number`.
How Frictionless tools treat that was not verified, so take care when comparing with Frictionless Table Schema.
6. **Package tree:** `docs/concepts.md` omits `suspects.jsonl` from its package tree, and [spec/12 §12.1](../spec/12-package.md#121-files) lists it.
7. **RFC 6962 is obsolete:** spec/08 §8.5 cites it; RFC 9162 obsoletes it (see above).
8. **Status:** The guides describe the formats as one set.
Of the 29 formats, 11 are `provisional`, including `scienting-dataset/1`, which the datasets paragraphs in `docs/concepts.md` present without that qualifier.
Of the 32 formats, 13 are `provisional`, including `scienting-dataset/1`, which the datasets paragraphs in `docs/concepts.md` present without that qualifier.
*The distinguishing combination, and what it costs.*
paperspec puts in one package permanent sentence- and cell-level IDs, five hashes per node, anchors with explicit resolution outcomes and drift, append-only evidence-gated corrections, signed fidelity receipts with spot-check bounds, per-witness rights with AI-use axes, and value-to-cell anchors.