feat: minted keys, VRF seeds, RFC 9162 proofs, history roots, tag URIs and a transparency log
Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
openalex in same_as, else the folded title, year and family name. A
collision lengthens the later key by two characters; a key is minted
once and never recomputed. A document key adds .si, .v<seq> or .<type>
for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
every node and record ID was re-minted and the docs rewritten to match.
The validator checks the document key against the work key and version
(doc.key.work) and the hash part against the identifier (doc.key.suffix).
vectors/keys.jsonl pins the cases paperstack's tests pin.
Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
implemented in tools/oracle/vrf.py in plain Python and reproduced
against the RFC's Example 16; the sample ranks by the VRF output. The
earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
verify_inclusion, consistency_proof, verify_consistency) with
scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
checkpoints with key hashes as transparency logs share them, the rule
that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).
Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
the checker refuses another version.
Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
checkpoint records; a check that every ID and hash the prose quotes
exists in the fixtures or vectors; three more broken packages
(history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
section, media-type registration and OCI distribution notes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
307 files changed
+2698
-1497
No files matched your search
+31
-31
@@ -41,10 +41,10 @@ printf '%s' 'The two parameter sets agree to within 2% for every bond.' | sha256
|
||||
175bfc8867816a9761fd5bb9ffc956df6d9e7f203870961a5485fc1912cb663a -
|
||||
```
|
||||
|
||||
The same value is in the package, in the `nodes.jsonl` of `hale2026harmonic`:
|
||||
The same value is in the package, in the `nodes.jsonl` of `hale2026harmonic-damb`:
|
||||
|
||||
```sh
|
||||
grep -o '"id":"j06tfhn".*' fixtures/packages/hale2026harmonic/nodes.jsonl | grep -o '"text_hash":"[0-9a-f]*"'
|
||||
grep -o '"id":"v3063qv".*' fixtures/packages/hale2026harmonic-damb/nodes.jsonl | grep -o '"text_hash":"[0-9a-f]*"'
|
||||
```
|
||||
|
||||
```text
|
||||
@@ -58,14 +58,14 @@ A package is a folder of JSON (JavaScript Object Notation, a plain-text format f
|
||||
`MANIFEST.json` lists every other file of a package with its SHA-256 and size, and the SHA-256 of the manifest itself pins the whole package ([chapter 12](../spec/12-package.md#123-the-manifest)).
|
||||
|
||||
```sh
|
||||
cd fixtures/packages/okafor2025open
|
||||
cd fixtures/packages/okafor2025solvent-acth
|
||||
sha256sum nodes.jsonl
|
||||
grep -o '"path":"nodes.jsonl","sha256":"[0-9a-f]*"' MANIFEST.json
|
||||
```
|
||||
|
||||
```text
|
||||
0fccd6492121fb91ebd1d32487e23ca5b30ad92627e7d3a0801d77549e1a2f94 nodes.jsonl
|
||||
"path":"nodes.jsonl","sha256":"0fccd6492121fb91ebd1d32487e23ca5b30ad92627e7d3a0801d77549e1a2f94"
|
||||
09c9b99eed97987e51dd0f85461ad9ad8768a0769ba7facfb7e0879d742660a3 nodes.jsonl
|
||||
"path":"nodes.jsonl","sha256":"09c9b99eed97987e51dd0f85461ad9ad8768a0769ba7facfb7e0879d742660a3"
|
||||
```
|
||||
|
||||
## Read a package with nothing but a JSON parser
|
||||
@@ -108,26 +108,26 @@ for n in nodes:
|
||||
```
|
||||
|
||||
```sh
|
||||
python3 outline.py fixtures/packages/hale2026harmonic
|
||||
python3 outline.py fixtures/packages/hale2026harmonic-damb
|
||||
```
|
||||
|
||||
```text
|
||||
bzqvbhd 1 Introduction
|
||||
qn08tj9 equation (1)
|
||||
f0tj6ca 2 Methods
|
||||
ajfvq4s figure Figure 1
|
||||
gm5z4rx table Table 1
|
||||
p36k8t6 1 Introduction
|
||||
tkxsmts equation (1)
|
||||
d400rz3 2 Methods
|
||||
v7cx6qs figure Figure 1
|
||||
hj4s7pb table Table 1
|
||||
```
|
||||
|
||||
## Follow an address
|
||||
|
||||
This shows how a citation points inside a sentence.
|
||||
An address names a node, and optionally a span of its text ([chapter 6](../spec/06-anchors.md#61-addresses)).
|
||||
`hale2026harmonic#f929eq9[29:44]` names code points 29 to 44 of sentence `f929eq9`.
|
||||
`hale2026harmonic-damb#hca6dc5[29:44]` names code points 29 to 44 of sentence `hca6dc5`.
|
||||
With `text()` from the script above:
|
||||
|
||||
```python
|
||||
print(text(by_id["f929eq9"])[29:44])
|
||||
print(text(by_id["hca6dc5"])[29:44])
|
||||
```
|
||||
|
||||
```text
|
||||
@@ -143,15 +143,15 @@ For text with characters outside the Basic Multilingual Plane the two counts dif
|
||||
This shows that the same packages can live in a database and be queried with SQL.
|
||||
|
||||
```sh
|
||||
pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic fixtures/packages/okafor2025open fixtures/packages/smithlab2027charges
|
||||
pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic-damb fixtures/packages/okafor2025solvent-acth fixtures/packages/writer2027fixed-ynr7.v3
|
||||
```
|
||||
|
||||
Without pixi, run `python3 tools/load.py` with the same arguments.
|
||||
|
||||
```text
|
||||
hale2026harmonic: 75 nodes
|
||||
okafor2025open: 15 nodes
|
||||
smithlab2027charges: 15 nodes
|
||||
hale2026harmonic-damb: 75 nodes
|
||||
okafor2025solvent-acth: 15 nodes
|
||||
writer2027fixed-ynr7.v3: 15 nodes
|
||||
```
|
||||
|
||||
The tables are those of [sql/doc.sql](../sql/doc.sql) and [sql/store.sql](../sql/store.sql), which run unchanged on Dolt and PostgreSQL as well ([chapter 13](../spec/13-database.md)); `tools/check.py` exercises them on SQLite only, and this repository has not yet run them on Dolt or PostgreSQL.
|
||||
@@ -163,14 +163,14 @@ import json, sqlite3
|
||||
db = sqlite3.connect("lab.db")
|
||||
|
||||
# Sentences the converter left below F2. Receipts can raise them; see below.
|
||||
for row in db.execute("SELECT id, fidelity FROM nodes WHERE doc_key = 'hale2026harmonic' "
|
||||
for row in db.execute("SELECT id, fidelity FROM nodes WHERE doc_key = 'hale2026harmonic-damb' "
|
||||
"AND kind = 'sentence' AND fidelity IN ('F0', 'F1') ORDER BY seq"):
|
||||
print(row)
|
||||
|
||||
# Every sentence that cites reference 2: the citation graph is sentence-level.
|
||||
for row in db.execute("SELECT c.sentence FROM citations c JOIN refs r "
|
||||
"ON r.doc_key = c.doc_key AND r.node = c.ref_node "
|
||||
"WHERE c.doc_key = 'hale2026harmonic' AND r.label = '2'"):
|
||||
"WHERE c.doc_key = 'hale2026harmonic-damb' AND r.label = '2'"):
|
||||
print(row)
|
||||
|
||||
# Who signed what.
|
||||
@@ -180,26 +180,26 @@ for row in db.execute("SELECT r.signer, r.scope, COUNT(*) FROM receipts r "
|
||||
|
||||
# Table 1's body, row by row.
|
||||
rows = db.execute("SELECT r.id FROM nodes r JOIN nodes b ON b.doc_key = r.doc_key AND b.id = r.parent "
|
||||
"WHERE r.doc_key = 'hale2026harmonic' AND r.kind = 'row' AND b.kind = 'table-body' "
|
||||
"WHERE r.doc_key = 'hale2026harmonic-damb' AND r.kind = 'row' AND b.kind = 'table-body' "
|
||||
"ORDER BY r.seq").fetchall()
|
||||
for (row,) in rows:
|
||||
cells = db.execute("SELECT content FROM nodes WHERE doc_key = 'hale2026harmonic' AND parent = ? ORDER BY ord",
|
||||
cells = db.execute("SELECT content FROM nodes WHERE doc_key = 'hale2026harmonic-damb' AND parent = ? ORDER BY ord",
|
||||
(row,)).fetchall()
|
||||
print(" | ".join("".join(r.get("text", "") for r in json.loads(c)) for (c,) in cells))
|
||||
```
|
||||
|
||||
```text
|
||||
('e4fkseg', 'F1')
|
||||
('r1gacps', 'F0')
|
||||
('eajq93t', 'F0')
|
||||
('f929eq9',)
|
||||
('ec5jc93', 'F1')
|
||||
('a4jrz2t', 'F0')
|
||||
('xjns57z', 'F0')
|
||||
('hca6dc5',)
|
||||
('ana', 'targeted:lode', 3)
|
||||
('ana', 'targeted:lode', 23)
|
||||
('ana', 'section:bzqvbhd', 2)
|
||||
('ana', 'section:p36k8t6', 2)
|
||||
('cara', 'page:2', 3)
|
||||
('ana', 'targeted:captions', 3)
|
||||
('ben', 'qa:eqgbbpwb6725v', 10)
|
||||
('ben', 'qa:gena76nkb6cnc', 3)
|
||||
('ben', 'qa:y2zw8ctg5s8dp', 10)
|
||||
('ben', 'qa:gx2s84z9jf9xv', 3)
|
||||
Bond parameters
|
||||
CT–CT | 310.0 | 1.526 | 305.2 | 1.5260
|
||||
CT–HC | 340.0 | 1.090 | 338.7 | 1.0900
|
||||
@@ -208,7 +208,7 @@ OW–HW | 553.0 | 0.9572 | 548.1 | 0.9570
|
||||
|
||||
The first query reads the level the converter recorded, not the final one.
|
||||
A receipt is a signed record that a reviewer checked certain nodes against the page.
|
||||
Receipts by `ana` raise `e4fkseg` and `r1gacps` to F3; `eajq93t` stays at F0 because the only receipt that covered it, by `cara`, was revoked.
|
||||
Receipts by `ana` raise `ec5jc93` and `a4jrz2t` to F3; `xjns57z` stays at F0 because the only receipt that covered it, by `cara`, was revoked.
|
||||
To compute the final level, check each receipt's signature, revocations and spot-checks, as [chapter 8](../spec/08-fidelity.md#82-levels) describes.
|
||||
The `scope` column of the third query shows the value `targeted:lode`, which is data in the fixture, copied here exactly as the query prints it.
|
||||
|
||||
@@ -239,11 +239,11 @@ This shows what the checks catch.
|
||||
Change one digit in a fixture and run the checks:
|
||||
|
||||
```sh
|
||||
sed -i 's/1718 cm/1719 cm/' fixtures/packages/okafor2025open/nodes.jsonl
|
||||
sed -i 's/1718 cm/1719 cm/' fixtures/packages/okafor2025solvent-acth/nodes.jsonl
|
||||
pixi run quick
|
||||
```
|
||||
|
||||
The checker reports three failures: the SHA-256 of `nodes.jsonl` no longer matches the manifest, unloading the package from the database no longer gives the same bytes, and the package hash differs after unloading.
|
||||
Restore the file with `git checkout fixtures/packages/okafor2025open/nodes.jsonl`.
|
||||
Restore the file with `git checkout fixtures/packages/okafor2025solvent-acth/nodes.jsonl`.
|
||||
The broken packages in [fixtures/invalid/](../fixtures/invalid/) are this kind of change, each with the rule it breaks.
|
||||
`git checkout` restores a tracked file, so run this only in a clone you can reset.
|
||||
Reference in new issue
Block a user