infra/paperspec
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: minted keys, VRF seeds, RFC 9162 proofs, history roots, tag URIs and a transparency log

Browse Source 
Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
  Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
  where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
  openalex in same_as, else the folded title, year and family name. A
  collision lengthens the later key by two characters; a key is minted
  once and never recomputed. A document key adds .si, .v<seq> or .<type>
  for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
  okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
  every node and record ID was re-minted and the docs rewritten to match.
  The validator checks the document key against the work key and version
  (doc.key.work) and the hash part against the identifier (doc.key.suffix).
  vectors/keys.jsonl pins the cases paperstack's tests pin.

Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
  implemented in tools/oracle/vrf.py in plain Python and reproduced
  against the RFC's Example 16; the sample ranks by the VRF output. The
  earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
  verify_inclusion, consistency_proof, verify_consistency) with
  scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
  the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
  checkpoints with key hashes as transparency logs share them, the rule
  that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).

Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
  namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
  the checker refuses another version.

Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
  checkpoint records; a check that every ID and hash the prose quotes
  exists in the fixtures or vectors; three more broken packages
  (history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
  keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
  section, media-type registration and OCI distribution notes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-14 23:21:57 -04:00
1 parent 76d76353d9
commit a09d158e16
307 files changed
+2698 -1497
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

schemas/README.md
+9 -8
View File
@@ -1,6 +1,6 @@
# Schemas
This folder holds the JSON (JavaScript Object Notation) Schemas (draft 2020-12) for every file and record that the paperspec formats (paperspec is the specification this repository holds) define: 39 schema files.
This folder holds the JSON (JavaScript Object Notation) Schemas (draft 2020-12) for every file and record that the paperspec formats (paperspec is the specification this repository holds) define: 42 schema files.
A JSON Schema is a file, itself written in JSON, that states what a valid JSON file looks like.
Newcomers can read it as a form: it lists the fields a record must have and what kind of value each field takes.
Implementers will find the loading recipe and the traps under "Validating" and "Writing schemas".
@@ -23,22 +23,23 @@ The number after each name is the chapter of `spec/` that defines the file.
| A package | `manifest.schema.json` (12), `document.schema.json` (2, 12), `node.schema.json` (3), `runs.schema.json` (3), `witnesses.schema.json` (2), `provenance.schema.json` (12), `page.schema.json` (7), `location.schema.json` (7), `reference.schema.json` (10), `id-entry.schema.json` (4), `correction.schema.json` (11), `receipt.schema.json` (8), `revocation.schema.json` (8), `roster.schema.json` (8), `suspect.schema.json` (5), `mathml.schema.json` (12) |
| Pointing into documents | `anchor.schema.json` (6) |
| Rights | `rights.schema.json` (14), `policy.schema.json` (14), `stages.schema.json` (14), `backend.schema.json` (14), `profile.schema.json` (14), `stamps.schema.json` (7) |
| Review | `qa-report.schema.json` (8), `spot-check.schema.json` (8), `checklist.schema.json` (8) |
| Review | `qa-report.schema.json` (8), `spot-check.schema.json` (8), `checklist.schema.json` (8), `proof.schema.json` (8, 11) |
| The transparency log | `log-entry.schema.json` (23), `checkpoint.schema.json` (23) |
| Annotation and teaching | `layer.schema.json` (18), `annotation.schema.json` (18), `exercise.schema.json` (19), `submission.schema.json` (19), `grade.schema.json` (19) |
| Data | `dataset.schema.json` (15), `value-anchor.schema.json` (15), `digitization.schema.json` (16), `view.schema.json` (17) |
| Teams and drafts | `project.schema.json` (20), `snapshot.schema.json` (21), `draft-ids.schema.json` (21) |
A record that travels on its own carries a `format` such as `scienting-anchor/1`. [registry/formats.json](../registry/formats.json) maps each format to its schema, its chapter and its status.
The formats registry lists 29 formats: 18 are `candidate` (complete, with vectors and fixtures, ready to implement) and 11 are `provisional` (complete; expected to change as implementations gain experience), as defined in `spec/00-introduction.md` §0.4.
The provisional formats are `scienting-project/1`, `scienting-bundle/1`, `scienting-dataset/1`, `scienting-digitization/1`, `scienting-view/1`, `scienting-snapshot/1`, `scienting-draft-ids/1`, `scienting-exercise/1`, `scienting-submission/1`, `scienting-grade/1` and `scienting-checklist/1`.
The Data schemas for datasets, digitizations and views, the Teams and drafts schemas, the three teaching schemas and `checklist.schema.json` belong to provisional formats.
The formats registry lists 32 formats: 19 are `candidate` (complete, with vectors and fixtures, ready to implement) and 13 are `provisional` (complete; expected to change as implementations gain experience), as defined in `spec/00-introduction.md` §0.4.
The provisional formats are `scienting-project/1`, `scienting-bundle/1`, `scienting-dataset/1`, `scienting-digitization/1`, `scienting-view/1`, `scienting-snapshot/1`, `scienting-draft-ids/1`, `scienting-exercise/1`, `scienting-submission/1`, `scienting-grade/1`, `scienting-checklist/1`, `scienting-log-entry/1` and `scienting-checkpoint/1`.
The Data schemas for datasets, digitizations and views, the Teams and drafts schemas, the three teaching schemas, `checklist.schema.json` and the two log schemas belong to provisional formats.
The rest belong to `candidate` formats or to no format of their own.
The bundle has no JSON Schema: its registry entry points to `sql/bundle-sqlite.sql`.
Of the 39 schemas, 11 have no format of their own because each describes a line or part of a larger file: `common`, `node`, `runs`, `witnesses`, `provenance`, `location`, `reference`, `id-entry`, `mathml`, `annotation` and `value-anchor`.
Of the schemas that do have one, 25 require a `format` property; `spot-check.schema.json` describes a signing payload rather than a file.
The others (`page`, `draft-ids` and `view`) describe a line of a file or a Vega-Lite specification (a JSON description of a chart), and the record itself does not need to carry the identifier.
Of the 42 schemas, 11 have no format of their own because each describes a line or part of a larger file: `common`, `node`, `runs`, `witnesses`, `provenance`, `location`, `reference`, `id-entry`, `mathml`, `annotation` and `value-anchor`.
Of the schemas that do have one, 27 require a `format` property; `spot-check.schema.json` describes the input a VRF proof is made over rather than a file.
The others (`page`, `draft-ids`, `view` and `log-entry`) describe a line of a file, a Vega-Lite specification (a JSON description of a chart) or an item of a tree, and the record itself does not need to carry the identifier.
## Validating
schemas/anchor.schema.json
+1 -1
View File
@@ -12,7 +12,7 @@
"$ref": "common.schema.json#/$defs/key"
},
"doc": {
"$ref": "common.schema.json#/$defs/key"
"$ref": "common.schema.json#/$defs/documentKey"
},
"node": {
"$ref": "common.schema.json#/$defs/nodeRef"
schemas/annotation.schema.json
+6 -6
View File
@@ -15,14 +15,14 @@
},
"id": {
"type": "string",
"pattern": "^urn:scienting:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
"pattern": "^tag:scient\\.ing,2026:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
},
"type": {
"const": "Annotation"
},
"layer": {
"type": "string",
"pattern": "^urn:scienting:layer:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
"pattern": "^tag:scient\\.ing,2026:layer:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
},
"motivation": {
"enum": [
@@ -528,7 +528,7 @@
},
"source": {
"type": "string",
"pattern": "^urn:scienting:doc:[a-z0-9][a-z0-9._-]*/[a-z0-9][a-z0-9._-]*$"
"pattern": "^tag:scient\\.ing,2026:doc:[a-z0-9][a-z0-9._-]*/[a-z][a-z0-9]*-[0-9a-hjkmnp-tv-z]{4,51}(\\.[a-z0-9][a-z0-9-]*)*$"
},
"selector": {
"type": "array",
@@ -593,7 +593,7 @@
},
"source": {
"type": "string",
"pattern": "^urn:scienting:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
"pattern": "^tag:scient\\.ing,2026:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
}
},
"required": [
@@ -616,7 +616,7 @@
},
"source": {
"type": "string",
"pattern": "^urn:scienting:doc:[a-z0-9][a-z0-9._-]*/[a-z0-9][a-z0-9._-]*$"
"pattern": "^tag:scient\\.ing,2026:doc:[a-z0-9][a-z0-9._-]*/[a-z][a-z0-9]*-[0-9a-hjkmnp-tv-z]{4,51}(\\.[a-z0-9][a-z0-9-]*)*$"
},
"selector": {
"type": "array",
@@ -681,7 +681,7 @@
},
"source": {
"type": "string",
"pattern": "^urn:scienting:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
"pattern": "^tag:scient\\.ing,2026:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
}
},
"required": [
schemas/checkpoint.schema.json
+62
View File
@@ -0,0 +1,62 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://paperspec.scient.ing/schemas/checkpoint.schema.json",
"title": "Log checkpoint",
"description": "scienting-checkpoint/1: a signed checkpoint of a team's transparency log, as the parsed form of its signed note; spec/23-log.md.",
"type": "object",
"properties": {
"format": {
"const": "scienting-checkpoint/1"
},
"origin": {
"type": "string",
"pattern": "^[^\\n]+$",
"minLength": 1
},
"size": {
"type": "integer",
"minimum": 0
},
"root": {
"$ref": "common.schema.json#/$defs/hash"
},
"signatures": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"$ref": "common.schema.json#/$defs/key"
},
"key_hash": {
"type": "string",
"pattern": "^[0-9a-f]{8}$"
},
"value": {
"$ref": "common.schema.json#/$defs/base64"
}
},
"required": [
"name",
"key_hash",
"value"
],
"additionalProperties": false
},
"minItems": 1
},
"note": {
"type": "string",
"minLength": 1
}
},
"required": [
"format",
"origin",
"size",
"root",
"signatures",
"note"
],
"additionalProperties": false
}
schemas/common.schema.json
+22 -6
View File
@@ -25,7 +25,20 @@
"key": {
"type": "string",
"pattern": "^[a-z0-9][a-z0-9._-]*$",
"maxLength": 128
"maxLength": 128,
"description": "A store, project or source key, or a legacy document key kept as an alias."
},
"workKey": {
"type": "string",
"pattern": "^[a-z][a-z0-9]*-[0-9a-hjkmnp-tv-z]{4,51}$",
"maxLength": 128,
"description": "A Scholar-style part, a hyphen and the hash of the work's identity (spec/02 §2.4)."
},
"documentKey": {
"type": "string",
"pattern": "^[a-z][a-z0-9]*-[0-9a-hjkmnp-tv-z]{4,51}(\\.[a-z0-9][a-z0-9-]*)*$",
"maxLength": 128,
"description": "A work key, with a dotted suffix for anything other than the version of record."
},
"alias": {
"type": "string",
@@ -45,8 +58,9 @@
},
"qualifiedKey": {
"type": "string",
"pattern": "^([a-z0-9][a-z0-9._-]*/)?[a-z0-9][a-z0-9._-]*$",
"maxLength": 257
"pattern": "^([a-z0-9][a-z0-9._-]*/)?[a-z][a-z0-9]*-[0-9a-hjkmnp-tv-z]{4,51}(\\.[a-z0-9][a-z0-9-]*)*$",
"maxLength": 257,
"description": "A document key, optionally qualified by its store."
},
"datasetId": {
"type": "string",
@@ -55,7 +69,7 @@
},
"address": {
"type": "string",
"pattern": "^([a-z0-9][a-z0-9._-]*/)?[a-z0-9][a-z0-9._-]*#[a-z0-9][a-z0-9._-]*(@[0-9a-f]{6})?(\\[(0|[1-9][0-9]*):(0|[1-9][0-9]*)\\])?$"
"pattern": "^([a-z0-9][a-z0-9._-]*/)?[a-z][a-z0-9]*-[0-9a-hjkmnp-tv-z]{4,51}(\\.[a-z0-9][a-z0-9-]*)*#[a-z0-9][a-z0-9._-]*(@[0-9a-f]{6})?(\\[(0|[1-9][0-9]*):(0|[1-9][0-9]*)\\])?$"
},
"time": {
"type": "string",
@@ -209,7 +223,9 @@
"ed25519",
"ssh",
"gpg",
"sigstore"
"sigstore",
"cose",
"ml-dsa-65"
]
},
"key": {
@@ -530,7 +546,7 @@
"properties": {
"id": {
"type": "string",
"pattern": "^urn:scienting:(key|person|software):[^ ]+$"
"pattern": "^tag:scient\\.ing,2026:(key|person|software):[^ ]+$"
},
"type": {
"enum": [
schemas/document.schema.json
+7 -3
View File
@@ -9,11 +9,14 @@
"const": "scienting-doc/1"
},
"key": {
"$ref": "common.schema.json#/$defs/key"
"$ref": "common.schema.json#/$defs/documentKey"
},
"store": {
"$ref": "common.schema.json#/$defs/key"
},
"history": {
"$ref": "common.schema.json#/$defs/hash"
},
"aliases": {
"type": "array",
"items": {
@@ -39,13 +42,13 @@
"type": "object",
"properties": {
"key": {
"$ref": "common.schema.json#/$defs/key"
"$ref": "common.schema.json#/$defs/workKey"
},
"same_as": {
"type": "array",
"items": {
"type": "string",
"pattern": "^(doi|arxiv|isbn|pmid|pmcid|url|lode):[^ ]+$"
"pattern": "^(doi|arxiv|isbn|pmid|pmcid|openalex|url|lode):[^ ]+$"
},
"uniqueItems": true
}
@@ -220,6 +223,7 @@
"hashing",
"root",
"meta_hash",
"history",
"rev",
"counts",
"fidelity"
schemas/grade.schema.json
+1 -1
View File
@@ -13,7 +13,7 @@
},
"exercise": {
"type": "string",
"pattern": "^urn:scienting:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
"pattern": "^tag:scient\\.ing,2026:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
},
"score": {
"$ref": "common.schema.json#/$defs/decimal"
schemas/layer.schema.json
+1 -1
View File
@@ -18,7 +18,7 @@
},
"id": {
"type": "string",
"pattern": "^urn:scienting:layer:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
"pattern": "^tag:scient\\.ing,2026:layer:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
},
"type": {
"const": "AnnotationCollection"
schemas/log-entry.schema.json
+24
View File
@@ -0,0 +1,24 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://paperspec.scient.ing/schemas/log-entry.schema.json",
"title": "Log entry",
"description": "scienting-log-entry/1: one item of a team's transparency log, [kind, id, hash]: the kind of record, its ID (or the store for a roster) and its record hash; spec/23-log.md.",
"type": "array",
"prefixItems": [
{
"enum": [
"receipt",
"revocation",
"roster"
]
},
{
"$ref": "common.schema.json#/$defs/key"
},
{
"$ref": "common.schema.json#/$defs/hash"
}
],
"minItems": 3,
"maxItems": 3
}
schemas/manifest.schema.json
+1 -1
View File
@@ -12,7 +12,7 @@
"const": "scienting-doc/1"
},
"key": {
"$ref": "common.schema.json#/$defs/key"
"$ref": "common.schema.json#/$defs/documentKey"
},
"files": {
"type": "array",
schemas/project.schema.json
+1 -1
View File
@@ -148,7 +148,7 @@
"type": "array",
"items": {
"type": "string",
"pattern": "^urn:scienting:layer:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
"pattern": "^tag:scient\\.ing,2026:layer:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
}
},
"datasets": {
schemas/proof.schema.json
+94
View File
@@ -0,0 +1,94 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://paperspec.scient.ing/schemas/proof.schema.json",
"title": "Merkle proof",
"description": "scienting-proof/1: an RFC 9162 inclusion proof of one item of a receipt, or a consistency proof between two revisions of a document's history; spec/08-fidelity.md §8.5, spec/11-corrections.md §11.5.",
"type": "object",
"properties": {
"format": {
"const": "scienting-proof/1"
},
"kind": {
"enum": [
"inclusion",
"consistency"
]
},
"receipt": {
"$ref": "common.schema.json#/$defs/recordId"
},
"doc": {
"$ref": "common.schema.json#/$defs/qualifiedKey"
},
"item": {
"type": "array",
"minItems": 1
},
"index": {
"type": "integer",
"minimum": 0
},
"size": {
"type": "integer",
"minimum": 1
},
"root": {
"$ref": "common.schema.json#/$defs/hash"
},
"from_rev": {
"type": "integer",
"minimum": 0
},
"to_rev": {
"type": "integer",
"minimum": 0
},
"from_root": {
"$ref": "common.schema.json#/$defs/hash"
},
"to_root": {
"$ref": "common.schema.json#/$defs/hash"
},
"path": {
"type": "array",
"items": {
"$ref": "common.schema.json#/$defs/hash"
}
}
},
"required": [
"format",
"kind",
"path"
],
"additionalProperties": false,
"allOf": [
{
"if": {
"properties": {
"kind": {
"const": "inclusion"
}
}
},
"then": {
"required": [
"receipt",
"item",
"index",
"size",
"root"
]
},
"else": {
"required": [
"doc",
"from_rev",
"to_rev",
"from_root",
"to_root"
]
}
}
]
}
schemas/receipt.schema.json
+1 -1
View File
@@ -12,7 +12,7 @@
"$ref": "common.schema.json#/$defs/recordId"
},
"doc": {
"$ref": "common.schema.json#/$defs/key"
"$ref": "common.schema.json#/$defs/documentKey"
},
"rev": {
"type": "integer",
schemas/roster.schema.json
+3 -1
View File
@@ -54,7 +54,9 @@
"ed25519",
"ssh",
"gpg",
"sigstore"
"sigstore",
"cose",
"ml-dsa-65"
]
},
"public": {
schemas/submission.schema.json
+1 -1
View File
@@ -13,7 +13,7 @@
},
"exercise": {
"type": "string",
"pattern": "^urn:scienting:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
"pattern": "^tag:scient\\.ing,2026:annotation:[a-z0-9][a-z0-9._-]*/[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$"
},
"doc": {
"$ref": "common.schema.json#/$defs/qualifiedKey"