feat: minted keys, VRF seeds, RFC 9162 proofs, history roots, tag URIs and a transparency log
Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
openalex in same_as, else the folded title, year and family name. A
collision lengthens the later key by two characters; a key is minted
once and never recomputed. A document key adds .si, .v<seq> or .<type>
for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
every node and record ID was re-minted and the docs rewritten to match.
The validator checks the document key against the work key and version
(doc.key.work) and the hash part against the identifier (doc.key.suffix).
vectors/keys.jsonl pins the cases paperstack's tests pin.
Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
implemented in tools/oracle/vrf.py in plain Python and reproduced
against the RFC's Example 16; the sample ranks by the VRF output. The
earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
verify_inclusion, consistency_proof, verify_consistency) with
scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
checkpoints with key hashes as transparency logs share them, the rule
that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).
Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
the checker refuses another version.
Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
checkpoint records; a check that every ID and hash the prose quotes
exists in the fixtures or vectors; three more broken packages
(history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
section, media-type registration and OCI distribution notes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
307 files changed
+2698
-1497
No files matched your search
@@ -21,14 +21,15 @@ Every file is JSON (a plain-text format for structured data) or JSON Lines (one
|
||||
A package is the unit these formats store a paper in: a folder or a zip file of such files holding the text as nodes (sections, sentences, cells, equations), page geometry, references, corrections, review receipts and the list of suspects ([spec/12](spec/12-package.md)).
|
||||
|
||||
```text
|
||||
cornell1995second#k3m9q2a@4f1a0c[12:57]
|
||||
└──── document ──┘└ node ┘└ pin ┘└span┘
|
||||
cornell1995second-saqk#k3m9q2a@4f1a0c[12:57]
|
||||
└────── document ────┘└ node ┘└ pin ┘└span┘
|
||||
```
|
||||
|
||||
A hash is a short fingerprint computed from text: the same text always gives the same fingerprint, and changing one character changes the hash.
|
||||
The string above names code points 12 to 57 of one sentence, as it read when its hash began `4f1a0c`.
|
||||
The node ID `k3m9q2a` encodes no page, position or kind.
|
||||
It is minted once from the document key, the node's kind, its text hash and its occurrence, and then frozen.
|
||||
The document key's last four characters, `saqk`, are the start of a hash of the paper's DOI, so two papers that a citation key alone would name alike never share a key ([spec/02 §2.4](spec/02-model.md#24-work-and-document-keys)).
|
||||
|
||||
Fixing a typo elsewhere in the paper leaves the ID alone, and converting the same input again mints the same IDs.
|
||||
A teammate's download of the same PDF, stamped differently by the publisher, has the same page fingerprints.
|
||||
@@ -53,7 +54,7 @@ The address grammar is in [spec/06 §6.1](spec/06-anchors.md#61-addresses), and
|
||||
|
||||
| Folder | Holds |
|
||||
| --- | --- |
|
||||
| [spec/](spec/README.md) | The specification, in 23 chapters. Normative. |
|
||||
| [spec/](spec/README.md) | The specification, in 24 chapters. Normative. |
|
||||
| [schemas/](schemas/README.md) | JSON Schemas, which state what a valid file or record looks like, for every file and record. |
|
||||
| [sql/](sql/README.md) | Tables for SQLite (a database kept in one file), Dolt and PostgreSQL, and the SQLite bundle with full-text search. |
|
||||
| [context/](context/README.md) | The JSON-LD context (a file that gives JSON terms shared meanings) that makes annotation layers W3C Web Annotations. |
|
||||
@@ -87,7 +88,7 @@ Without pixi, install those two with pip and run `python3 tools/check.py`.
|
||||
To explore a worked package, load the example paper (a synthetic chemistry paper with 12 corrections) into SQLite and query it; [docs/examples.md](docs/examples.md) has queries to start from.
|
||||
|
||||
```sh
|
||||
pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic
|
||||
pixi run load --sqlite lab.db fixtures/packages/hale2026harmonic-damb
|
||||
```
|
||||
|
||||
## Format names
|
||||
@@ -101,8 +102,8 @@ Schemas use the identifier base `https://paperspec.scient.ing/schemas/`.
|
||||
## Status and limits
|
||||
|
||||
This is version 0.1.0.
|
||||
Of the 29 formats, 18 are `candidate` and 11 are `provisional`.
|
||||
The `provisional` formats are project, bundle, dataset, digitization, view, snapshot, draft-ids, exercise, submission, grade and checklist; where those are described, they are marked as such.
|
||||
Of the 32 formats, 19 are `candidate` and 13 are `provisional`.
|
||||
The `provisional` formats are project, bundle, dataset, digitization, view, snapshot, draft-ids, exercise, submission, grade, checklist, log-entry and checkpoint; where those are described, they are marked as such.
|
||||
The repository includes a reference checker, test vectors and fixtures.
|
||||
|
||||
The formats are built for a team-scale corpus of hundreds of documents, not hundreds of millions of papers.
|
||||
|
||||
Reference in new issue
Block a user