alexmandClaude
a09d158e16
Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
openalex in same_as, else the folded title, year and family name. A
collision lengthens the later key by two characters; a key is minted
once and never recomputed. A document key adds .si, .v<seq> or .<type>
for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
every node and record ID was re-minted and the docs rewritten to match.
The validator checks the document key against the work key and version
(doc.key.work) and the hash part against the identifier (doc.key.suffix).
vectors/keys.jsonl pins the cases paperstack's tests pin.
Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
implemented in tools/oracle/vrf.py in plain Python and reproduced
against the RFC's Example 16; the sample ranks by the VRF output. The
earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
verify_inclusion, consistency_proof, verify_consistency) with
scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
checkpoints with key hashes as transparency logs share them, the rule
that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).
Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
the checker refuses another version.
Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
checkpoint records; a check that every ID and hash the prose quotes
exists in the fixtures or vectors; three more broken packages
(history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
section, media-type registration and OCI distribution notes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
4.7 KiB
12. The package
A document travels as a package: a folder named by its key, or the same folder zipped.
12.1 Files
| Path | Required | Holds |
|---|---|---|
MANIFEST.json |
Yes | The manifest (12.3). |
document.json |
Yes | The document (2.2). |
nodes.jsonl |
Yes | Every node, in document order (3). |
witnesses.json |
Yes | Every witness, with its rights (2.3). |
provenance.json |
Yes | Tools and versions, backends, cached readings, stages and where each ran. |
pages.jsonl |
No | Every page of every witness (7.3). |
locations.jsonl |
No | Geometry per node and page (7.4). |
references.jsonl |
No | Parsed references and their resolution (10). |
corrections.jsonl |
No | Every correction, in order (11). |
receipts.jsonl |
No | Receipts and revocations, sorted by at and then id (8). |
ids.jsonl |
No | Retired IDs and legacy aliases (4.4). |
suspects.jsonl |
No | Unresolved suspects, sorted by id (5.7). |
roster.json |
No | The roster its receipts need (8.7). |
assets/ |
No | Images and crops, as <sha256>.<extension>, present or listed only. |
datasets/<id>/ |
No | Datasets (15). |
derived/ |
No | Rebuildable renderings, never authoritative (12.7). |
An optional JSON Lines file with no records is omitted. Every file is in the canonical form of 1.9.
12.2 Paths
Paths use /, are relative, have no empty, . or .. segment, no backslash and no NUL, and are already normalized.
No two paths are the same once case-folded, so a package reads alike on every file system.
A package folder carries no symbolic links, and a consumer refuses one that does, as it refuses a package whose files exceed its own size limit.
In a zip, entries are UTF-8 named, sit under one top-level folder named by the key, are stored or deflated, carry no symbolic links, and are unique; a consumer refuses an archive that breaks any of these, or whose expanded size exceeds its own limit.
A producer that wants byte-identical archives writes entries sorted by path, dated 1980-01-01 00:00, with no extra fields; fixtures/packages/tiny2026tiny-j4qm.sct.zip is such an archive.
12.3 The manifest
MANIFEST.json, scienting-package/1, is {format, doc, key, files}: the package format, scienting-doc/1, the document key, and every other file as {path, sha256, size}, sorted by the UTF-8 bytes of the path.
An asset whose bytes stay in the asset store is listed with present: false; nothing else may be.
The SHA-256 of MANIFEST.json pins the whole package.
12.4 Provenance
provenance.json holds tools ({name, version}), and optionally backends, readings (each cached reading's ID, witness, backend and SHA-256), stages (each stage's name, tool, input hashes, output hash, the processor where it ran, and the uses it ran under) and a session.
Every stage that uses a model or an outside service records where it ran (14).
12.5 Text policies
| Policy | Means |
|---|---|
full |
Every leaf keeps its content. |
checks-only |
Every leaf has withheld: true and no content; panels drop their figure text; references keep their resolution but no CSL; there are no corrections, suspects or datasets; and the hashing scheme is keyed: hmac-team between the team's machines and hkdf-pages outside the team (5.8). |
A checks-only package is how a team shares its annotations on a paywalled paper without sharing the paper.
Its text lives in the team's private asset store, and only key holders can check its values.
fixtures/packages/hale2026harmonic-damb.checks-only is such an export of hale2026harmonic-damb to another machine of the team, under the team key in fixtures/keys/.
12.6 Assets
Assets are named by SHA-256, so every read is rehashed and a regenerated crop is a different asset.
Bytes that live only in the asset store are listed in the manifest with present: false, with their size.
12.7 Derived files
derived/ holds what a pinned tool makes from the package, such as derived/mathml.jsonl, whose lines are {node, run, tool, mathml} with run null for an equation node.
Nothing derived is content, and a consumer may delete and rebuild it.
12.8 Media type and extension
A zipped package has the extension .sct.zip and the media type application/vnd.scienting.doc+zip.