feat: minted keys, VRF seeds, RFC 9162 proofs, history roots, tag URIs and a transparency log

Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
  Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
  where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
  openalex in same_as, else the folded title, year and family name. A
  collision lengthens the later key by two characters; a key is minted
  once and never recomputed. A document key adds .si, .v<seq> or .<type>
  for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
  okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
  every node and record ID was re-minted and the docs rewritten to match.
  The validator checks the document key against the work key and version
  (doc.key.work) and the hash part against the identifier (doc.key.suffix).
  vectors/keys.jsonl pins the cases paperstack's tests pin.

Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
  implemented in tools/oracle/vrf.py in plain Python and reproduced
  against the RFC's Example 16; the sample ranks by the VRF output. The
  earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
  verify_inclusion, consistency_proof, verify_consistency) with
  scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
  the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
  checkpoints with key hashes as transparency logs share them, the rule
  that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).

Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
  namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
  the checker refuses another version.

Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
  checkpoint records; a check that every ID and hash the prose quotes
  exists in the fixtures or vectors; three more broken packages
  (history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
  keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
  section, media-type registration and OCI distribution notes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-14 23:21:57 -04:00
1 parent 76d76353d9
commit a09d158e16
307 files changed
+2698 -1497

No files matched your search

+45 -5
View File
@@ -9,7 +9,7 @@ Implementers find the chapter that defines each rule.
Experts find the cost of each choice and, where a neighboring standard chose differently, how it differs (the full comparison is in [comparison.md](comparison.md)).
The formats are at version 0.1.0.
Of the 29 in `registry/formats.json`, 18 are `candidate` and 11 are `provisional`, and the provisional ones are named where they appear.
Of the 29 in `registry/formats.json`, 19 are `candidate` and 13 are `provisional`, and the provisional ones are named where they appear.
The repository is paperspec, at <https://git.scient.ing/infra/paperspec>.
The `scienting-` prefix in format names such as `scienting-anchor/1` is the formats' namespace, separate from the repository's name.
@@ -36,6 +36,18 @@ The cost is that two conversions that read a sentence differently by one charact
Everything a team builds points at IDs: notes, claims, dataset values, citations in drafts.
A merge records its successor, a deletion records that there is none, and legacy IDs live on as aliases, all in `ids.jsonl` ([spec/04 §4.3](../spec/04-identity.md#43-permanence-retirement-and-aliases)).
### Work keys carry a hash
A citation key such as `vaswani2017attention` is what readers say, and it collides: a preprint and its published version, or two papers by one first author in one year with the same first title word, would share it.
Four characters of a hash of the work's identity, its DOI when it has one, settle the collision and let any tool holding the DOI recompute the key.
The key is minted once and never recomputed, so a corrected title cannot change a name someone has cited; when two works still share four characters, the later one gets six ([spec/02 §2.4](../spec/02-model.md#24-work-and-document-keys)).
The rule is the one the studio's reading-list tool already uses, so a paper keeps its key from the reading list to the store.
### Identifiers are tag URIs
Layers, annotations and documents name themselves with `tag:scient.ing,2026:` URIs (RFC 4151) rather than `urn:` names, because a URN namespace must be registered with IANA and this one is not.
A tag URI needs only a domain and a date, and these identifiers sit inside hashed annotations, so the cheap time to choose was before release.
### Revisions are counted, not timed
Anchors are made on different machines and packages are imported later, and clocks disagree.
@@ -97,6 +109,11 @@ An offset counts Unicode code points, the numbered characters of Unicode, into t
Every stored offset is still exact when `node_hash` is unchanged, and a renderer can slice the stored text without normalizing it first.
Only the three text values, `text_hash`, `match_hash` and `align_key`, normalize.
### The character data is pinned to one Unicode version
Normalization and General Categories change between Unicode versions, and both enter every hash.
The chapters pin Unicode 16.0, the reference tools refuse to run on another version, and moving to a later one is a change to every text-hashing format ([spec/01 §1.4](../spec/01-conventions.md#14-text-normalization-for-hashing)).
### Hashes that leave the team are keyed
A plain hash of "310.0" is recovered by hashing guesses, and overlapping anchors can rebuild passages.
@@ -156,6 +173,11 @@ Readings drop spans, roles and header paths, not only words, and those need a co
Drift is computed from the records alone, with no need to replay history, because each record carries `path`, `to_path` and `touched` ([spec/11 §11.3](../spec/11-corrections.md#113-applied-fields)).
### The history has a root
Corrections are append-only in principle, but nothing in a package proved that a later revision extends an earlier one rather than rewriting it.
A Merkle root over the corrections in `document.json`, with RFC 9162 consistency proofs between revisions, does ([spec/11 §11.5](../spec/11-corrections.md#115-the-history-root)).
### Digits need image evidence
A wrong digit changes a result without looking wrong, so no digit changes without a crop of the page as evidence, and no leaf outside the correction's `touched` list may end up with different digits ([spec/11 §11.4](../spec/11-corrections.md#114-the-gate)).
@@ -182,11 +204,17 @@ The receipt tree follows [RFC 6962](https://www.rfc-editor.org/rfc/rfc6962), who
The specification makes a receipt with no items invalid, where the RFC hashes the empty list ([spec/08 §8.5](../spec/08-fidelity.md#85-merkle-roots)).
The tree is not the one behind `node_hash`, which hashes each node's `own_hash` with its children's `node_hash` values.
### Spot-check samples come from a signature nobody can steer
### Spot-check seeds are VRF proofs
A seed the reviewer could influence could be ground until the sample fell on careful work.
The checker's deterministic Ed25519 signature over the receipt is unpredictable to the reviewer and unchoosable by the checker, because the checker's key was registered for spot-checks before the original was signed.
Anyone can recompute the sample ([spec/08 §8.8](../spec/08-fidelity.md#88-spot-checks)).
A seed the reviewer could influence could be ground until the sample fell on careful work, and a seed the checker could vary could be ground by the checker.
An earlier draft used the checker's Ed25519 signature and called it unchoosable because signing is deterministic; that holds only for an honest checker, since a verifier accepts any valid signature and a dishonest one can sign with any nonce.
A verifiable random function (RFC 9381) has exactly one valid proof for a key and an input, which is the property the rule needs, and its keys are ordinary Ed25519 keys, so the roster needs nothing new.
Anyone can verify the proof and recompute the sample ([spec/08 §8.8](../spec/08-fidelity.md#88-spot-checks)).
### A receipt can be proven without being shown
The receipt tree follows RFC 9162 so that an inclusion proof, a dozen hashes, shows a buyer that one cell was proofread without handing over the receipt or the other nodes ([spec/08 §8.5](../spec/08-fidelity.md#85-merkle-roots)).
A transparency log of receipts, revocations and rosters ([spec/23](../spec/23-log.md)) uses the same tree and the checkpoint format transparency logs share, so that a receipt's time and the roster that governed it stop being self-asserted.
### Error rates are reported as bounds
@@ -270,6 +298,18 @@ These are settled for now, and a future version may revisit them.
| Which drift threshold, spot-check rate and digit rule should teams use? | By default, drift is high above a quarter of the leaves involved, and spot-checks sample 5% of a receipt's nodes, at least ten. The digit rule is `all`, `anchored-sections` or `none`; each is set per project. |
| How should rights combine when a document comes from witnesses held on different bases? | The most restrictive witness governs each use. |
## Looking ahead
Choices made so that the formats still serve in the 2030s, and the ones deliberately left open.
- Signature schemes `cose` and `ml-dsa-65` are reserved: COSE so that the IETF's SCITT transparency services can verify receipts, ML-DSA (FIPS 204) for the time Ed25519 is no longer considered safe.
A consumer treats an unimplemented scheme as invalid, so adopting either is a minor version ([spec/08 §8.4](../spec/08-fidelity.md#84-record-hashes-and-signatures)).
- Every hash is SHA-256, and the hashing scheme names only how a value is keyed, not the function.
A hash function change would be a new major version of every format; the formats accept that rather than carrying agility they cannot test.
- Roster keys are named by short keys, not by decentralized identifiers; a `did:key` form could be added as a key property without a new version.
- A correction's author may be a model and its run; a fuller provenance for model output (the model's identity, weights and prompt by hash) is left to the project that first needs it.
- A transparency log is specified but no deployment exists yet; its tile layout is left to the first one ([spec/23](../spec/23-log.md)).
## Known limits
- Notation drops spaces and text keeps them, so `$T = 300$` from a LaTeX source and "T = 300" from a PDF disagree, and become a `word` suspect for review.