feat: minted keys, VRF seeds, RFC 9162 proofs, history roots, tag URIs and a transparency log

Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
  Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
  where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
  openalex in same_as, else the folded title, year and family name. A
  collision lengthens the later key by two characters; a key is minted
  once and never recomputed. A document key adds .si, .v<seq> or .<type>
  for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
  okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
  every node and record ID was re-minted and the docs rewritten to match.
  The validator checks the document key against the work key and version
  (doc.key.work) and the hash part against the identifier (doc.key.suffix).
  vectors/keys.jsonl pins the cases paperstack's tests pin.

Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
  implemented in tools/oracle/vrf.py in plain Python and reproduced
  against the RFC's Example 16; the sample ranks by the VRF output. The
  earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
  verify_inclusion, consistency_proof, verify_consistency) with
  scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
  the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
  checkpoints with key hashes as transparency logs share them, the rule
  that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).

Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
  namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
  the checker refuses another version.

Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
  checkpoint records; a check that every ID and hash the prose quotes
  exists in the fixtures or vectors; three more broken packages
  (history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
  keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
  section, media-type registration and OCI distribution notes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-14 23:21:57 -04:00
1 parent 76d76353d9
commit a09d158e16
307 files changed
+2698 -1497

No files matched your search

+9 -5
View File
@@ -53,7 +53,7 @@ The procedure is the same for every file:
Find the inputs and the expected outputs of the case.
The table below says which fields are which, and the chapter it links defines the rule.
Only six files, `alignment`, `corrections`, `grading`, `resolve`, `rights` and `targets`, put the expected outputs under `expect`.
In the other fifteen the expected outputs are fields beside the inputs, such as `output`, `sha256`, `canonical` or `parsed`.
In the other nineteen the expected outputs are fields beside the inputs, such as `output`, `sha256`, `canonical` or `parsed`.
4.
Compute each output from the inputs with your implementation.
5.
@@ -78,7 +78,7 @@ Some files change the procedure:
Several files also mix more than one kind of case, told apart by their fields.
`ids.jsonl` has minting cases and letter-first cases, `keyed.jsonl` and `signatures.jsonl` have a case for each derivation or record, `spotcheck.jsonl` has sample-size cases and one full sample, and `rights.jsonl` has stage checks (a `stage` field), backend checks (a `backend` field) and one combining case (neither).
The reference checker in [tools/](../tools/README.md) runs all twenty-one files, together with the rest of the repository's checks:
The reference checker in [tools/](../tools/README.md) runs all twenty-five files, together with the rest of the repository's checks:
```sh
python3 tools/check.py --quick
@@ -123,18 +123,22 @@ The table below uses a few terms that the chapters define:
| `rights.jsonl` | Stage checks with policies and expiry, the combining rule, and backend conditions. | [14](../spec/14-rights.md) |
| `grading.jsonl` | Exercises, submissions and the grades they earn. | [19.4](../spec/19-exercises.md#194-grading) |
| `targets.jsonl` | The target set of targeted F3 for the example paper under each digit rule, given the anchored nodes. | [8.10](../spec/08-fidelity.md#810-targeted-f3) |
| `keys.jsonl` | Work keys: Scholar parts, identities, hash suffixes and document keys, with accented names, missing years and a collision. | [2.4](../spec/02-model.md#24-work-and-document-keys) |
| `proofs.jsonl` | RFC 9162 inclusion paths for every item of small trees, consistency proofs for every prefix, the empty root, and the example paper's history. | [8.5](../spec/08-fidelity.md#85-merkle-roots), [11.5](../spec/11-corrections.md#115-the-history-root) |
| `vrf.jsonl` | The VRF of spot-check seeds: RFC 9381's Example 16, two more inputs for its key, the example paper's seed, and proofs that must not verify (`beta` null). Bytes are hex. | [8.8](../spec/08-fidelity.md#88-spot-checks) |
| `checkpoint.jsonl` | A transparency-log checkpoint: the entries, their root, the signed note and its signers. | [23](../spec/23-log.md) |
Resolve those paths against the files in that package only.
## What the vectors prove, and what they do not
The specification, and the vectors with it, are at version 0.1.0 ([0.3](../spec/00-introduction.md#03-format-identifiers-and-versions)).
The specification has 29 formats: 18 are `candidate` and 11 are `provisional` in [registry/formats.json](../registry/formats.json).
The specification has 32 formats: 19 are `candidate` and 13 are `provisional` in [registry/formats.json](../registry/formats.json).
A candidate format changes only with a recorded reason and a new or changed vector ([22.5](../spec/22-conformance.md#225-stability)).
A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in `tools/oracle/`.
That code computed the expected outputs, and `pixi run build` regenerates them ([tools/](../tools/README.md)).
The vectors hold 221 cases in all, some of them small: `hashes.jsonl` has one document, `alignment.jsonl` one pair of readings, `offsets.jsonl` and `signatures.jsonl` two cases each.
The vectors hold 306 cases in all, some of them small: `hashes.jsonl` has one document, `alignment.jsonl` one pair of readings, `offsets.jsonl` and `signatures.jsonl` two cases each.
The vectors do not show the following.
@@ -148,7 +152,7 @@ The vectors do not show the following.
- `canonical-json.jsonl` has five cases, none for the repository's rule that a lone surrogate, `NaN` or an infinity has no canonical form.
- `merkle.jsonl` has roots for 1, 2, 3, 4, 5 and 7 items.
A receipt with no items is invalid ([8.5](../spec/08-fidelity.md#85-merkle-roots)), so no case has an empty list, although the RFCs it follows define a hash for an empty list ([RFC 6962](https://www.rfc-editor.org/rfc/rfc6962), [RFC 9162](https://www.rfc-editor.org/rfc/rfc9162)).
- Eleven formats are `provisional`, and only `grading.jsonl` carries any of their identifiers (exercise, submission and grade).
- Thirteen formats are `provisional`, and only `grading.jsonl` carries any of their identifiers (exercise, submission and grade).
The chapters of the others define values that no vector pins, such as a dataset's content hash ([15](../spec/15-datasets.md)), the hash that confirms a digitization ([16](../spec/16-digitization.md)) and a checklist's content hash ([8.11](../spec/08-fidelity.md#811-checklists)).
They are expected to change as implementations gain experience ([0.4](../spec/00-introduction.md#04-status)).
- Reading a PDF: how text lines are extracted from a PDF is outside the specification; `fingerprints.jsonl` supplies the lines, so it tests the fingerprint function only ([7.1](../spec/07-geometry.md#71-page-fingerprints)).