feat: minted keys, VRF seeds, RFC 9162 proofs, history roots, tag URIs and a transparency log
Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
openalex in same_as, else the folded title, year and family name. A
collision lengthens the later key by two characters; a key is minted
once and never recomputed. A document key adds .si, .v<seq> or .<type>
for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
every node and record ID was re-minted and the docs rewritten to match.
The validator checks the document key against the work key and version
(doc.key.work) and the hash part against the identifier (doc.key.suffix).
vectors/keys.jsonl pins the cases paperstack's tests pin.
Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
implemented in tools/oracle/vrf.py in plain Python and reproduced
against the RFC's Example 16; the sample ranks by the VRF output. The
earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
verify_inclusion, consistency_proof, verify_consistency) with
scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
checkpoints with key hashes as transparency logs share them, the rule
that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).
Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
the checker refuses another version.
Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
checkpoint records; a check that every ID and hash the prose quotes
exists in the fixtures or vectors; three more broken packages
(history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
section, media-type registration and OCI distribution notes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
307 files changed
+2698
-1497
No files matched your search
@@ -44,7 +44,7 @@ The prefix `scienting-` is the namespace of the formats.
|
||||
The namespace is not the name of this repository, paperspec, and formats are named after what they hold.
|
||||
|
||||
A file or record that can travel on its own carries its identifier in a `format` property.
|
||||
The anchor in `fixtures/records/anchor-leaf.json` begins `{"doc":"hale2026harmonic","format":"scienting-anchor/1", ...`.
|
||||
The anchor in `fixtures/records/anchor-leaf.json` begins `{"doc":"hale2026harmonic-damb","format":"scienting-anchor/1", ...`.
|
||||
Records inside a package file whose format the manifest fixes, such as the lines of `nodes.jsonl`, do not repeat it.
|
||||
|
||||
A change that makes a valid file invalid, or that changes a computed value, is a breaking change and needs a new major version, with a migration.
|
||||
@@ -56,7 +56,7 @@ The specification as a whole is versioned with semantic versioning; this is vers
|
||||
## 0.4 Status
|
||||
|
||||
Each format has one of three statuses, listed in [registry/formats.json](../registry/formats.json).
|
||||
In version 0.1.0 the registry lists 29 formats: 18 are `candidate` and 11 are `provisional`; none is `reserved`.
|
||||
In version 0.1.0 the registry lists 32 formats: 19 are `candidate` and 13 are `provisional`; none is `reserved`.
|
||||
|
||||
| Status | Meaning |
|
||||
| --- | --- |
|
||||
|
||||
@@ -43,6 +43,9 @@ U+200B ZERO WIDTH SPACE is not whitespace.
|
||||
Stored text is kept exactly as witnessed; normalization applies only when a value is computed.
|
||||
The vectors in `normalize.jsonl` cover the cases, including the angstrom sign, which NFC turns into Å.
|
||||
|
||||
Normalization forms, General Categories and the `White_Space` property are those of Unicode 16.0.
|
||||
A later Unicode version can change a decomposition or a category, and with it a hash, so an implementation states the Unicode version its character data comes from, the reference tools refuse to run on another, and moving the formats to a later version is a change to every format that hashes text ([0.3](00-introduction.md#03-format-identifiers-and-versions)).
|
||||
|
||||
## 1.5 Character offsets
|
||||
|
||||
Offsets count Unicode code points, never UTF-16 code units or bytes.
|
||||
@@ -67,7 +70,9 @@ No file carries the time it was written, unless the time is part of the record,
|
||||
| --- | --- | --- |
|
||||
| Node ID | Seven characters of lowercase Crockford base32, the first a letter. | `^[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6}$` |
|
||||
| Record ID | 7 to 26 such characters, the first a letter. Derived records use 13. | `^[a-hjkmnp-tv-z][0-9a-hjkmnp-tv-z]{6,25}$` |
|
||||
| Key | A document, work, store, project or source key. At most 128 characters. | `^[a-z0-9][a-z0-9._-]*$` |
|
||||
| Work key | A Scholar-style part, a hyphen and 4 to 51 characters of Crockford base32 ([2.4](02-model.md#24-work-and-document-keys)). At most 128 characters. | `^[a-z][a-z0-9]*-[0-9a-hjkmnp-tv-z]{4,51}$` |
|
||||
| Document key | A work key, with a dotted suffix for a version other than the version of record. At most 128 characters. | `^[a-z][a-z0-9]*-[0-9a-hjkmnp-tv-z]{4,51}(\.[a-z0-9][a-z0-9-]*)*$` |
|
||||
| Key | A store, project or source key, or a legacy document key kept as an alias. At most 128 characters. | `^[a-z0-9][a-z0-9._-]*$` |
|
||||
| Alias | A legacy ID; it always holds a `.`, `_` or `-`, so it never looks like a node ID. | `^[a-z0-9][a-z0-9._-]*[._-][a-z0-9._-]*$` |
|
||||
| Dataset ID | `ds-` and a key. | `^ds-[a-z0-9][a-z0-9._-]*$` |
|
||||
|
||||
|
||||
+33
-2
@@ -4,12 +4,12 @@
|
||||
|
||||
| Object | Is | Identified by |
|
||||
| --- | --- | --- |
|
||||
| Work | The intellectual work, across all its versions. | A key, and `same_as` identifiers: `doi:`, `arxiv:`, `isbn:`, `pmid:`, `pmcid:`, `url:` or `lode:`. |
|
||||
| Work | The intellectual work, across all its versions. | A work key ([2.4](#24-work-and-document-keys)), and `same_as` identifiers: `doi:`, `arxiv:`, `isbn:`, `pmid:`, `pmcid:`, `openalex:`, `url:` or `lode:`. |
|
||||
| Version | One expression of the work: a preprint, an accepted manuscript, the version of record, an erratum, Supporting Information, a draft. | The work's key and a version type, with an optional sequence number and date. |
|
||||
| Witness | One file of a version: a PDF, JATS XML, a LaTeX source archive, an HTML page, an EPUB, a Typst project. | The SHA-256 of its bytes. |
|
||||
| Document | The text established for one version from one or more witnesses. | A key, and a root hash per revision. |
|
||||
|
||||
A document's key is its work's key, with a suffix for anything other than the version of record: `cornell1995second`, `cornell1995second.si`, `dong2026machine.v2`.
|
||||
A document's key is its work's key, with a suffix for anything other than the version of record: `cornell1995second-saqk`, `cornell1995second-saqk.si`, `dong2026machine-1ard.v2`.
|
||||
Supporting Information is its own document, with its own pages, numbering and references, related to its article by `supplement-of`.
|
||||
A textbook is a work whose chapters are documents related by `part-of`.
|
||||
A draft is a document of kind `draft`.
|
||||
@@ -36,6 +36,7 @@ A renamed document keeps its old keys in `aliases`.
|
||||
| `hashing` | Yes | How content values are computed: `{scheme: "plain"}`, a team HMAC or a page-derived key ([5.8](05-hashes.md#58-keyed-hashes)). |
|
||||
| `root` | Yes | The root node's `node_hash`. |
|
||||
| `meta_hash` | Yes | The metadata hash ([5.9](05-hashes.md#59-the-metadata-hash)). |
|
||||
| `history` | Yes | The history root: the root of the tree over the package's corrections ([11.5](11-corrections.md#115-the-history-root)). |
|
||||
| `rev` | Yes | The revision: the index of the last correction applied, or 0. |
|
||||
| `counts` | Yes | `{nodes, leaves}`. |
|
||||
| `fidelity` | Yes | `{lowest, counts}`: the lowest level among leaves, or null when there are none, and the number of leaves at each level, computed as [8.2](08-fidelity.md#82-levels) defines. |
|
||||
@@ -62,3 +63,33 @@ The schema is `document.schema.json`.
|
||||
The witness's bytes are not part of the package; they live in the team's asset store, by SHA-256.
|
||||
The SHA-256 is the exact identity of one file.
|
||||
Because publishers stamp each download, two copies of one paper rarely share it; page fingerprints identify their pages instead ([7](07-geometry.md)).
|
||||
|
||||
## 2.4 Work and document keys
|
||||
|
||||
A work key names a work the way a reader would, and never collides: a Scholar-style citation key, a hyphen, and the first characters of a hash of the work's identity.
|
||||
|
||||
```text
|
||||
vaswani2017attention-a241
|
||||
└─── Scholar part ──┘ └hash┘
|
||||
```
|
||||
|
||||
**The Scholar part** is built as Google Scholar builds its BibTeX keys, from the CSL-JSON `meta` of the version:
|
||||
|
||||
1. the first author's family name: `family`, or the part of a `literal` name before a comma, else its last word that is not a suffix such as Jr. or III; folded to ASCII with NFKD, lower-cased, letters only, so `Müller` gives `muller` and `O'Brien` gives `obrien`; no author, or a name with no Latin letters, gives `anon`;
|
||||
2. the year of `issued`, when it has four digits, else nothing;
|
||||
3. the first word of `title` that is not a stop word, folded the same way, letters and digits only, with apostrophes removed and other punctuation splitting words; no such word gives nothing.
|
||||
|
||||
The stop words are `a an the of on in for to and or at by with from as into onto about toward towards via is are be its it this that these those`.
|
||||
|
||||
**The identity** is the work's first identifier among `doi`, `arxiv`, `isbn`, `pmid`, `pmcid` and `openalex` in `same_as`, in that order, written `<kind>:<value>` with a DOI's ASCII letters lower-cased.
|
||||
A work with none has the identity `meta:<title words>|<year>|<family>` from the same folded parts.
|
||||
`url:` and `lode:` identifiers name copies and claims, not the work, and never enter the identity.
|
||||
|
||||
**The hash part** is the leading bits of SHA-256 of `scienting-key/1:` followed by the identity, in UTF-8, written in lowercase Crockford base32, five bits a character: four characters, or, when another work in the store already holds that key, six, eight and so on until the key is free.
|
||||
A key is minted once, when the work enters a store, and never recomputed: a corrected title or a year revised later leaves it alone, while the hash of an identifier still shows which work the key names.
|
||||
A work key matches `^[a-z][a-z0-9]*-[0-9a-hjkmnp-tv-z]{4,51}$`.
|
||||
|
||||
**The document key** is the work key for the version of record, and the work key with a suffix for any other version: `.si` for Supporting Information, `.v<seq>` for a preprint, submitted, accepted or draft version with a sequence number, else a dot and the version type: `cornell1995second-saqk`, `cornell1995second-saqk.si`, `dong2026machine-1ard.v2`.
|
||||
A validator checks that the document key is the work key with the version's suffix (`doc.key.work`), and that a key whose identity is an identifier carries that identifier's hash (`doc.key.suffix`); a metadata identity may have been corrected since the key was minted, so it is not held to the key.
|
||||
Store, project and source keys keep the form `^[a-z0-9][a-z0-9._-]*$`, as do legacy document keys kept in `aliases`.
|
||||
`keys.jsonl` pins the Scholar parts, identities, hashes and document keys, with accented names, name suffixes, missing years and authors, and the lengthening that settles a collision.
|
||||
+1
-1
@@ -174,7 +174,7 @@ So every content value that leaves a team's store is keyed.
|
||||
| A public layer, or an export outside the team on a work whose text is withheld | `{"scheme": "hkdf-pages", "context": …, "pages": […]}` | A key derived from the witness's page fingerprints. |
|
||||
|
||||
The destination chooses the scheme, not the text policy.
|
||||
A checks-only package ([12.5](12-package.md#125-text-policies)) sent between the team's own machines is keyed with the team key, as `hale2026harmonic.checks-only` is; sent to anyone outside the team, it is keyed with the page key.
|
||||
A checks-only package ([12.5](12-package.md#125-text-policies)) sent between the team's own machines is keyed with the team key, as `hale2026harmonic-damb.checks-only` is; sent to anyone outside the team, it is keyed with the page key.
|
||||
|
||||
A keyed scheme replaces SHA-256 with HMAC-SHA-256 under its key in every content value, including the Merkle construction of `node_hash`, and in `meta_hash`.
|
||||
|
||||
|
||||
+11
-8
@@ -6,22 +6,25 @@ An address names a node, optionally pinned to a hash and narrowed to a span:
|
||||
|
||||
```abnf
|
||||
address = [store "/"] key "#" node ["@" pin] ["[" start ":" end "]"]
|
||||
store = key
|
||||
key = lowalnum *(lowalnum / "." / "_" / "-")
|
||||
node = key ; a node ID or an alias
|
||||
store = lowalnum *(lowalnum / "." / "_" / "-")
|
||||
key = workkey *("." lowalnum *(lowalnum / "-")) ; a document key (2.4)
|
||||
workkey = LOWER *lowalnum "-" 4*51BASE32
|
||||
node = lowalnum *(lowalnum / "." / "_" / "-") ; a node ID or an alias
|
||||
pin = 6HEXLOWER ; the first six digits of a node_hash
|
||||
start = "0" / (%x31-39 *DIGIT)
|
||||
end = "0" / (%x31-39 *DIGIT)
|
||||
lowalnum = %x61-7A / DIGIT
|
||||
lowalnum = LOWER / DIGIT
|
||||
LOWER = %x61-7A
|
||||
BASE32 = DIGIT / %x61-68 / %x6A-6B / %x6D-6E / %x70-74 / %x76-7A ; Crockford, lowercase
|
||||
```
|
||||
|
||||
`start` is not greater than `end`, and both are code-point offsets into the plain projection of the node in the pinned state.
|
||||
|
||||
| Example | Names |
|
||||
| --- | --- |
|
||||
| `cornell1995second#k3m9q2a` | A sentence, whatever its current text. |
|
||||
| `cornell1995second#k3m9q2a@4f1a0c` | That sentence as it read when its `node_hash` began `4f1a0c`. |
|
||||
| `cornell1995second#k3m9q2a@4f1a0c[12:57]` | Code points 12 to 57 of its plain projection in that state. |
|
||||
| `cornell1995second-saqk#k3m9q2a` | A sentence, whatever its current text. |
|
||||
| `cornell1995second-saqk#k3m9q2a@4f1a0c` | That sentence as it read when its `node_hash` began `4f1a0c`. |
|
||||
| `cornell1995second-saqk#k3m9q2a@4f1a0c[12:57]` | Code points 12 to 57 of its plain projection in that state. |
|
||||
| `smithlab/cornell1995second#k3m9q2a` | The same sentence in the `smithlab` store. |
|
||||
|
||||
On the web the node ID is the fragment: `…/smithlab/cornell1995second#k3m9q2a` lands on the sentence.
|
||||
@@ -102,7 +105,7 @@ The result is `{result, node, node_hash?, span?, drift?, via?, reason?}`: the ou
|
||||
A `moved` or `reattached` anchor is re-pinned to the live node, and the move is recorded.
|
||||
|
||||
Checks are stricter than readers: an extract is a span or leaf anchor and never accepts `changed-below`, because a check is about exact words.
|
||||
`resolve.jsonl` resolves seventeen anchors made at revision 0 against `fixtures/packages/hale2026harmonic` at revision 12, with every outcome.
|
||||
`resolve.jsonl` resolves seventeen anchors made at revision 0 against `fixtures/packages/hale2026harmonic-damb` at revision 12, with every outcome.
|
||||
|
||||
## 6.6 Drift
|
||||
|
||||
|
||||
+17
-7
@@ -61,7 +61,10 @@ Its signing payload is the canonical JSON of `{"format": <the record's format>,
|
||||
| `ssh` | An SSH signature of the payload in the namespace `scienting`, armored. |
|
||||
| `gpg` | A detached, armored OpenPGP signature of the payload. |
|
||||
| `sigstore` | A Sigstore bundle for the payload, as JSON text. |
|
||||
| `cose` | Reserved: a COSE_Sign1 message over the payload, in Base64, as the IETF SCITT work shapes signed statements, so other transparency services can verify a receipt. No vector yet. |
|
||||
| `ml-dsa-65` | Reserved: an ML-DSA-65 signature (FIPS 204) of the payload, in Base64, for the time Ed25519 is no longer considered safe. No vector yet. |
|
||||
|
||||
A consumer treats a record signed with a scheme it does not implement as invalid, so a reserved scheme can be adopted by a new minor version without any format changing.
|
||||
`signature.key` is the signer's roster key and equals `by`.
|
||||
Self-hosted deployments sign with team keys or a private Sigstore deployment; the public Sigstore instance, whose log permanently records each signer's identity, is for public layers only.
|
||||
`signatures.jsonl` verifies one receipt and the RFC 8032 test vector.
|
||||
@@ -72,7 +75,12 @@ A receipt's items are `["node", id, node_hash]` for each listed node, in order,
|
||||
The root follows RFC 6962: a leaf hash is SHA-256 of the byte 0x00 and the item's canonical JSON; an interior hash is SHA-256 of 0x01 and the two child hashes; a list of *n* > 1 items splits after the largest power of two less than *n*.
|
||||
RFC 6962 has since been replaced by RFC 9162, which defines the same tree.
|
||||
A root over one item is that item's leaf hash; a receipt with no items is invalid.
|
||||
The tree allows a proof that one node was covered without revealing the others; `merkle.jsonl` gives roots for one to seven items.
|
||||
`merkle.jsonl` gives roots for one to seven items.
|
||||
|
||||
The tree allows a proof that one node was covered without revealing the others.
|
||||
An inclusion proof, `scienting-proof/1` with `kind` `inclusion`, holds the `receipt`, the `item`, its `index` among the receipt's items, the `size` of the tree, the `root` and the `path`: the audit path of RFC 9162 §2.1.3.1, as hex hashes, leaf-most first.
|
||||
A verifier recomputes the root from the item's leaf hash and the path as RFC 9162 §2.1.3.2 describes and compares it with the receipt's root.
|
||||
`proofs.jsonl` gives the path for every item of trees of one to seven items, and `fixtures/records/proof-inclusion.json` proves one node of the example paper's first receipt.
|
||||
|
||||
## 8.6 Receipt status
|
||||
|
||||
@@ -101,14 +109,16 @@ The roster is meant to be the one a store verifies against; see the [rationale](
|
||||
## 8.8 Spot-checks
|
||||
|
||||
For each F3 receipt, a second reviewer checks a sample of its nodes and signs a `qa:` receipt.
|
||||
A seed the reviewer controls could be ground, by varying the receipt until the sample falls on careful work, so the seed is a signature nobody can steer:
|
||||
A seed the reviewer controls could be ground, by varying the receipt until the sample falls on careful work, and a seed the checker controls could be ground by the checker, so the seed is a value neither can steer:
|
||||
|
||||
1. The seed is the checker's Ed25519 signature over the canonical JSON of `{"format": "scienting-spot-check/1", "receipt": <the original's record hash>}`, made with a key registered in the roster with the `spot-check` purpose before the original was signed.
|
||||
Ed25519 signatures are deterministic, so the checker cannot choose it, and the reviewer cannot predict it.
|
||||
1. The seed is the checker's VRF proof, under RFC 9381's ECVRF-EDWARDS25519-SHA512-TAI, for the input that is the canonical JSON of `{"format": "scienting-spot-check/1", "receipt": <the original's record hash>}`, made with an Ed25519 key registered in the roster with the `spot-check` purpose before the original was signed.
|
||||
A verifiable random function has exactly one valid proof for a key and an input, where an Ed25519 signature has many, so the checker cannot try proofs until the sample suits, and the reviewer cannot predict the proof without the checker's private key.
|
||||
2. The sample size is the smaller of *n* and the larger of `minimum` and ⌈`rate` × *n*⌉, for the original's *n* nodes; the defaults are a rate of `0.05` and a minimum of 10.
|
||||
3. Rank the original's node IDs by the hexadecimal SHA-256 of the seed's 64 bytes followed by the ID's UTF-8, then by ID, and take the first ones.
|
||||
3. Rank the original's node IDs by the hexadecimal SHA-256 of the proof's VRF output, the 64 bytes `ECVRF_proof_to_hash` gives, followed by the ID's UTF-8, then by ID, and take the first ones.
|
||||
|
||||
The `qa` object records `of`, `key`, `seed` (Base64), `rate`, `minimum`, the `sample` sorted by ID, and the `errors` found, each `{node, kind?, note?}`.
|
||||
The `qa` object records `of`, `key`, `seed` (the 80-byte proof, Base64), `rate`, `minimum`, the `sample` sorted by ID, and the `errors` found, each `{node, kind?, note?}`.
|
||||
A verifier checks the proof against the checker's public key and the input before it recomputes the sample.
|
||||
`vrf.jsonl` pins the VRF on the RFC's own example and on the example paper's seed.
|
||||
The `qa:` receipt's `nodes` are the sampled nodes at the hashes the original lists, its `scope` is `qa:` and the original's ID, every error names a sampled node, and it is signed at or after the original.
|
||||
Anyone can recompute the sample; `spotcheck.jsonl` does.
|
||||
The payload's identifier, `scienting-spot-check/1`, is registered with the other formats so that no file ever carries it by mistake; it names a signing payload, never a file.
|
||||
@@ -137,7 +147,7 @@ Most of a paper is never relied on, so a team proofreads the target set: the nod
|
||||
|
||||
The set is reported in document order, each node with the first reason that applies: `anchored`, `equation` or `digits`.
|
||||
Neighbors shown for context are not in the set, and nothing new comes to rest on a node below F3 without the node joining the set.
|
||||
`targets.jsonl` gives the set of `fixtures/packages/hale2026harmonic` under each digit rule.
|
||||
`targets.jsonl` gives the set of `fixtures/packages/hale2026harmonic-damb` under each digit rule.
|
||||
|
||||
## 8.11 Checklists
|
||||
|
||||
|
||||
@@ -72,3 +72,11 @@ The gate needs only the document's current state, so a tool can run it on a prop
|
||||
Digits are characters of General Category Nd in the plain projection.
|
||||
A package's corrections are also checked as a whole: indexes run from 1 without gaps, each `id` derives from its record, `rev` equals their number, every retired ID is in `ids.jsonl`, and the last correction of each leaf ends at its current `node_hash`.
|
||||
`corrections.jsonl` in the vectors applies each operation, and each refusal, to a small document.
|
||||
|
||||
## 11.5 The history root
|
||||
|
||||
`document.json` carries `history`: the root of the tree of [8.5](08-fidelity.md#85-merkle-roots) whose items are `["correction", index, id]` for each correction in order, or SHA-256 of the empty string when there are none, which is how RFC 9162 hashes the empty tree.
|
||||
Because corrections are only appended, a consistency proof (RFC 9162 §2.1.4) between the roots at two revisions shows that the later history extends the earlier one rather than rewriting it.
|
||||
The proof is a `scienting-proof/1` with `kind` `consistency`, holding the `doc`, `from_rev`, `to_rev`, `from_root`, `to_root` and the `path` as hex hashes; a verifier applies RFC 9162 §2.1.4.2.
|
||||
A checks-only package carries no corrections, so its history is the empty root.
|
||||
`proofs.jsonl` gives consistency proofs for every prefix of small trees and for the example paper between revisions 8 and 12; `fixtures/records/proof-consistency.json` is the latter.
|
||||
+2
-2
@@ -32,7 +32,7 @@ Paths use `/`, are relative, have no empty, `.` or `..` segment, no backslash an
|
||||
No two paths are the same once case-folded, so a package reads alike on every file system.
|
||||
A package folder carries no symbolic links, and a consumer refuses one that does, as it refuses a package whose files exceed its own size limit.
|
||||
In a zip, entries are UTF-8 named, sit under one top-level folder named by the key, are stored or deflated, carry no symbolic links, and are unique; a consumer refuses an archive that breaks any of these, or whose expanded size exceeds its own limit.
|
||||
A producer that wants byte-identical archives writes entries sorted by path, dated 1980-01-01 00:00, with no extra fields; `fixtures/packages/tiny2026example.sct.zip` is such an archive.
|
||||
A producer that wants byte-identical archives writes entries sorted by path, dated 1980-01-01 00:00, with no extra fields; `fixtures/packages/tiny2026tiny-j4qm.sct.zip` is such an archive.
|
||||
|
||||
## 12.3 The manifest
|
||||
|
||||
@@ -54,7 +54,7 @@ Every stage that uses a model or an outside service records where it ran ([14](1
|
||||
|
||||
A checks-only package is how a team shares its annotations on a paywalled paper without sharing the paper.
|
||||
Its text lives in the team's private asset store, and only key holders can check its values.
|
||||
`fixtures/packages/hale2026harmonic.checks-only` is such an export of `hale2026harmonic` to another machine of the team, under the team key in `fixtures/keys/`.
|
||||
`fixtures/packages/hale2026harmonic-damb.checks-only` is such an export of `hale2026harmonic-damb` to another machine of the team, under the team key in `fixtures/keys/`.
|
||||
|
||||
## 12.6 Assets
|
||||
|
||||
|
||||
+1
-1
@@ -24,4 +24,4 @@ A digitized value never passes as reported, and its uncertainty is at least the
|
||||
|
||||
A dataset's content hash is SHA-256 of the canonical JSON of `{"format": "scienting-dataset/1", "files": [[path, sha256, size], …]}` over its files, sorted by path.
|
||||
A dataset can be a source of kind `dataset`, so a claim can rest on a value read from a figure, with that fact visible in its evidence.
|
||||
`fixtures/packages/hale2026harmonic/datasets/ds-hale-bond-params` reports Table 1, and the suite checks that every value resolves to the cell that prints it.
|
||||
`fixtures/packages/hale2026harmonic-damb/datasets/ds-hale-bond-params` reports Table 1, and the suite checks that every value resolves to the cell that prints it.
|
||||
+3
-3
@@ -12,7 +12,7 @@ If it is about reading this paper, it belongs in a layer, where it is attributed
|
||||
## 18.2 The layer
|
||||
|
||||
A layer is a folder with `layer.json` and `annotations.jsonl`.
|
||||
`layer.json` is an `AnnotationCollection` with the two contexts, `format`, an `id` of the form `urn:scienting:layer:<store>/<record id>`, a `label`, a `purpose`, an `audience` (`novice`, `student`, `expert` or `mixed`), a `visibility` (`private`, `team` or `public`), a `license` (CC BY 4.0 by default when public), its `creator`, `created`, its `project`, the `docs` it annotates and its `total`.
|
||||
`layer.json` is an `AnnotationCollection` with the two contexts, `format`, an `id` of the form `tag:scient.ing,2026:layer:<store>/<record id>`, a `label`, a `purpose`, an `audience` (`novice`, `student`, `expert` or `mixed`), a `visibility` (`private`, `team` or `public`), a `license` (CC BY 4.0 by default when public), its `creator`, `created`, its `project`, the `docs` it annotates and its `total`.
|
||||
A reader turns layers on and off, and the audience dial picks the level of glosses.
|
||||
|
||||
## 18.3 The annotation
|
||||
@@ -22,12 +22,12 @@ Each line of `annotations.jsonl` is a W3C `Annotation`:
|
||||
| Property | Holds |
|
||||
| --- | --- |
|
||||
| `@context` | `["http://www.w3.org/ns/anno.jsonld", "https://paperspec.scient.ing/context/scienting-1.jsonld"]`. |
|
||||
| `id` | `urn:scienting:annotation:<store>/<record id>` ([4.6](04-identity.md#46-record-ids)). |
|
||||
| `id` | `tag:scient.ing,2026:annotation:<store>/<record id>` ([4.6](04-identity.md#46-record-ids)). |
|
||||
| `type`, `layer` | `Annotation`, and the layer's ID. |
|
||||
| `motivation` | A W3C motivation: `commenting`, `describing`, `linking`, `questioning`, `assessing`, `replying` and the others. |
|
||||
| `creator`, `created`, `modified` | An agent `{id, type, name}`, with `type` `Person` or `Software`, and times. |
|
||||
| `body` | One body ([18.4](#184-bodies)). |
|
||||
| `target` | A `SpecificResource` whose `source` is `urn:scienting:doc:<store>/<key>` and whose `selector` lists an `AnchorSelector` holding a `scienting-anchor/1`, optionally followed by a W3C `TextQuoteSelector` for tools that do not know ours; or, for a reply, a `SpecificResource` whose source is another annotation. |
|
||||
| `target` | A `SpecificResource` whose `source` is `tag:scient.ing,2026:doc:<store>/<key>` and whose `selector` lists an `AnchorSelector` holding a `scienting-anchor/1`, optionally followed by a W3C `TextQuoteSelector` for tools that do not know ours; or, for a reply, a `SpecificResource` whose source is another annotation. |
|
||||
| `status` | `proposed`, `confirmed`, `rejected` or `withdrawn`. A model's annotation is `proposed` until a person confirms it. |
|
||||
| `receipt` | For a confirmed annotation, the receipt that covers it. |
|
||||
|
||||
|
||||
@@ -19,10 +19,10 @@ A validator checks, in order, and reports each failure as `path:line: rule: mess
|
||||
|
||||
1. The manifest: present, valid, sorted, every listed file present with its hash and size, nothing unlisted, paths safe and no symbolic links, required files present.
|
||||
2. Every file in canonical form, byte for byte, and every record against its schema.
|
||||
3. The tree ([3](03-nodes.md)), runs and references between nodes, and the counts.
|
||||
3. The tree ([3](03-nodes.md)), runs and references between nodes, the counts, and the keys ([2.4](02-model.md#24-work-and-document-keys)).
|
||||
4. The policy and hashing scheme ([12.5](12-package.md#125-text-policies)).
|
||||
5. The five values of every node, the root and `meta_hash`, when the text is present and the key, if any, is known.
|
||||
6. The ID table, the corrections, the witnesses, pages, locations and references, and every file's order.
|
||||
6. The ID table, the corrections and the history root, the witnesses, pages, locations and references, and every file's order.
|
||||
7. Receipts: IDs, roots, documents, node order, signatures against the package's roster, spot-check keys, seeds and samples, and the fidelity distribution.
|
||||
|
||||
## 22.3 Schemas
|
||||
@@ -43,6 +43,7 @@ Rule IDs are stable: a rule's meaning never changes, and a retired rule's ID is
|
||||
| `doc.hash.` | The five values, the root and the metadata hash. |
|
||||
| `doc.id.`, `doc.order.` | The ID table and file order. |
|
||||
| `doc.policy.` | Text policies and keyed hashing. |
|
||||
| `doc.key.`, `doc.counts`, `doc.history` | The keys, the counts and the history root of `document.json`. |
|
||||
| `doc.correction.`, `correction.` | A package's corrections, and the gate. |
|
||||
| `doc.receipt.`, `doc.qa.`, `doc.fidelity.` | Receipts, spot-checks and levels. |
|
||||
| `doc.witness.`, `doc.page.`, `doc.location.`, `doc.reference.` | Witnesses and geometry. |
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
# 23. The transparency log
|
||||
|
||||
A receipt's `at` is what its signer wrote, and a roster is a file a package carries.
|
||||
Neither proves that a receipt existed before the key that signed it was revoked, or that the roster a verifier holds is the one the team published.
|
||||
A team's transparency log settles both: an append-only record of every receipt, revocation and roster the team publishes, whose state is a Merkle tree root that the team signs and that witnesses may cosign.
|
||||
The log is the same tree as receipts and the correction history ([8.5](08-fidelity.md#85-merkle-roots), [11.5](11-corrections.md#115-the-history-root)), with the same proofs.
|
||||
Both formats of this chapter are `provisional` ([0.4](00-introduction.md#04-status)): the design follows the specifications that transparency logs share, and the first deployment will shape the details.
|
||||
|
||||
## 23.1 Entries
|
||||
|
||||
An entry, `scienting-log-entry/1`, is one item of the tree, `[kind, id, hash]`:
|
||||
|
||||
| `kind` | `id` | `hash` |
|
||||
| --- | --- | --- |
|
||||
| `receipt` | The receipt's record ID. | Its record hash ([8.4](08-fidelity.md#84-record-hashes-and-signatures)). |
|
||||
| `revocation` | The revocation's record ID. | Its record hash. |
|
||||
| `roster` | The store the roster is for. | SHA-256 of the canonical JSON of the roster. |
|
||||
|
||||
Entries are appended in the order the team publishes them and never removed.
|
||||
The leaf hash of an entry is SHA-256 of the byte 0x00 and its canonical JSON, the interior hashes follow RFC 9162, and the root over no entries is SHA-256 of the empty string, as for the history root.
|
||||
A roster is logged before any receipt a key it adds may sign, so a verifier can find the roster that governed a receipt by looking at the entries before it.
|
||||
|
||||
## 23.2 Checkpoints
|
||||
|
||||
A checkpoint, `scienting-checkpoint/1`, is a signed note in the format transparency logs share: an origin line, the tree size, the root hash in Base64, an empty line, and one signature line per signer.
|
||||
|
||||
```text
|
||||
scient.ing/log/smithlab
|
||||
41
|
||||
Q2xlYXIgdGV4dCBpcyBub3QgYSByb290IGhhc2guLi4uLi4uLi4=
|
||||
|
||||
— ana AAAAAAHZ…
|
||||
```
|
||||
|
||||
The origin names the log, and a team's logs are named `scient.ing/log/<store>` or under a domain the team controls.
|
||||
A signature line is an em dash (U+2014), a space, the signer's roster key, a space, and Base64 of the signer's four-byte key hash followed by the signature.
|
||||
The key hash is the first four bytes of SHA-256 of the signer's name, a newline, the byte 0x01 and the Ed25519 public key; the signature is Ed25519 over the note's first three lines, each ending in a newline.
|
||||
A note parsed into JSON has `format`, `origin`, `size`, `root` as hex, `signatures` as `{name, key_hash, value}` and the `note` text itself; the schema is `checkpoint.schema.json`.
|
||||
`checkpoint.jsonl` gives a checkpoint over the example paper's receipts and its verification.
|
||||
|
||||
A checkpoint may carry further signature lines from witnesses, independent parties that cosign only a checkpoint consistent with every earlier one they saw.
|
||||
A verifier that holds two checkpoints of one log checks a consistency proof between them ([11.5](11-corrections.md#115-the-history-root)); a log that cannot produce one has rewritten its past.
|
||||
|
||||
## 23.3 What a log settles
|
||||
|
||||
When a project names a log in its settings ([20](20-projects.md)), a receipt counts toward a level only when the log holds it, and the log's order replaces the records' own times in two rules:
|
||||
|
||||
1. A revocation of a signer's period revokes the receipts logged before the revocation that fall in the period; a receipt logged after a revocation of its key never counts, whatever its `at` says.
|
||||
2. The roster that governs a receipt is the latest roster entry before it.
|
||||
|
||||
A reader fetches the latest checkpoint, verifies its signatures against the roster and its consistency with the checkpoint it last saw, then verifies inclusion proofs for the receipts it relies on.
|
||||
Without a log the rules of [8.6](08-fidelity.md#86-receipt-status) stand, and the `at` of each record is taken at its word.
|
||||
|
||||
## 23.4 Serving a log
|
||||
|
||||
A log is served as tiles, the layout transparency logs share, so that clients fetch only the subtrees they need and a static file server can host a log.
|
||||
The tile layout and the exact URL scheme are left to the first deployment; this chapter fixes only the entries, the tree and the checkpoint, which any layout must preserve.
|
||||
+2
-1
@@ -18,7 +18,7 @@ Chapter numbers are those in the table below.
|
||||
| Anything at all | 0, 1, 3, 4 and 5: introduction, conventions, nodes, identity and the content values. |
|
||||
| A reader of packages | Then 2, 6, 7 and 12: the model, anchors, geometry and the package. |
|
||||
| A converter | Then 9, 10 and 11: figures and tables, references, and corrections with the gate. |
|
||||
| A store or a server | Then 8, 13 and 14: receipts, the database form, and rights. |
|
||||
| A store or a server | Then 8, 13, 14 and 23: receipts, the database form, rights, and the transparency log. |
|
||||
| A writer or reader of annotation, teaching or data files | Then 15 to 21. |
|
||||
| A validator | Chapter 22, with [registry/rules.json](../registry/rules.json). |
|
||||
|
||||
@@ -49,3 +49,4 @@ Chapter numbers are those in the table below.
|
||||
| [20. Projects](20-projects.md) | Members and roles, documents, reading paths, held layers, datasets, views and drafts, and settings |
|
||||
| [21. Drafts](21-drafts.md) | Drafts: snapshots and sentence-ID sidecars |
|
||||
| [22. Conformance](22-conformance.md) | Conformance, the suite and the rule catalog |
|
||||
| [23. The transparency log](23-log.md) | A team's log of receipts, revocations and rosters, its checkpoints, and what it settles |
|
||||
Reference in new issue
Block a user