alexmandClaude
a09d158e16
Keys (spec/02 §2.4), the rule paperstack settled on:
- A work key is a Scholar-style citation key, a hyphen and the first four
Crockford base32 characters of SHA-256("scienting-key/1:" + identity),
where the identity is the first of doi, arxiv, isbn, pmid, pmcid and
openalex in same_as, else the folded title, year and family name. A
collision lengthens the later key by two characters; a key is minted
once and never recomputed. A document key adds .si, .v<seq> or .<type>
for versions other than the version of record.
- Every fixture key is now minted (hale2026harmonic-damb,
okafor2025solvent-acth, writer2027fixed-ynr7.v3, tiny2026tiny-j4qm), so
every node and record ID was re-minted and the docs rewritten to match.
The validator checks the document key against the work key and version
(doc.key.work) and the hash part against the identifier (doc.key.suffix).
vectors/keys.jsonl pins the cases paperstack's tests pin.
Review:
- Spot-check seeds are ECVRF-EDWARDS25519-SHA512-TAI proofs (RFC 9381),
implemented in tools/oracle/vrf.py in plain Python and reproduced
against the RFC's Example 16; the sample ranks by the VRF output. The
earlier Ed25519 seed could be ground by a dishonest checker.
- RFC 9162 inclusion and consistency proofs (crypto.inclusion_path,
verify_inclusion, consistency_proof, verify_consistency) with
scienting-proof/1, vectors/proofs.jsonl and two fixture records.
- document.json carries `history`, the root over the corrections, with
the empty-tree hash for none (spec/11 §11.5, rule doc.history).
- A provisional transparency log (spec/23): log entries, signed-note
checkpoints with key hashes as transparency logs share them, the rule
that a logged revocation beats a self-asserted time, and vectors.
- Reserved signature schemes cose and ml-dsa-65 (spec/08 §8.4).
Identifiers and text:
- urn:scienting: becomes tag:scient.ing,2026: (RFC 4151), since the URN
namespace was never registered.
- Unicode 16.0 is pinned for normalization and categories (spec/01 §1.4);
the checker refuses another version.
Checking:
- check.py: keys, proofs, VRF and checkpoint vectors; the proof and
checkpoint records; a check that every ID and hash the prose quotes
exists in the fixtures or vectors; three more broken packages
(history-wrong, key-suffix, key-work).
- stress.py: key, VRF, proof and checkpoint properties.
- Docs: a keys section in the concepts and FAQ, rationale entries for
keys, VRF, proofs, tag URIs and the Unicode pin, a "Looking ahead"
section, media-type registration and OCI distribution notes.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
90 lines
5.8 KiB
Markdown
90 lines
5.8 KiB
Markdown
# 0. Introduction
|
|
|
|
This chapter is for three readers.
|
|
A newcomer, who knows what a scientific paper and a citation are, finds what the specification covers and how to read it.
|
|
An implementer, who will write a reader, writer, converter or checker, finds the rules for conformance, identifiers, versions and status.
|
|
An expert, who knows the neighboring standards, finds the status of each format and how the specification relates to those standards.
|
|
|
|
## 0.1 Scope
|
|
|
|
Tools that read, correct, annotate and check papers can work together only if they agree on the files they exchange.
|
|
The specification defines that data: documents and their packages, anchors, receipts, rights records, layers, exercises, datasets, digitizations, views, projects and bundles.
|
|
A package is a document stored as a set of files; an anchor is a reference to a place in a document that stays checkable when the document changes.
|
|
|
|
It defines data, not programs.
|
|
A conforming file can be read, checked and loaded without any particular program, and every rule is stated so that a second implementation, in any language, computes the same bytes.
|
|
|
|
The specification does not replace the standards around it.
|
|
JATS ([ANSI/NISO Z39.96-2024](https://www.niso.org/publications/z3996-2024-jats)) remains the format for publishers and archives to exchange journal articles; here it is only one kind of input to a document.
|
|
Layers are [W3C Web Annotations](https://www.w3.org/TR/annotation-model/) with a custom selector for the anchor. [docs/comparison.md](../docs/comparison.md) compares the specification with the standards near it and says where each is the better choice.
|
|
|
|
The [guides](../docs/README.md) explain what each format is for, and the [rationale](../docs/rationale.md) why its rules are what they are.
|
|
|
|
## 0.2 Conformance language
|
|
|
|
The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, RECOMMENDED, MAY and OPTIONAL are to be interpreted as described in RFC 2119 and RFC 8174 when, and only when, they appear in capitals.
|
|
|
|
The suite is the set of test vectors (inputs paired with the outputs every implementation must compute) and fixtures (example files, valid and invalid) in the repository. [22](22-conformance.md) describes it.
|
|
|
|
The specification distinguishes three classes of implementation.
|
|
|
|
| Class | Does | Must |
|
|
| --- | --- | --- |
|
|
| Producer | Writes files in a format. | Write only files that validate, with values computed as specified. |
|
|
| Consumer | Reads files in a format. | Accept every valid file, and refuse every file the suite lists as invalid, naming the rule. |
|
|
| Validator | Decides whether a file is valid. | Check every rule in [22](22-conformance.md) that the file's contents allow it to check. |
|
|
|
|
An implementation states, for each format, which versions it reads and writes.
|
|
|
|
## 0.3 Format identifiers and versions
|
|
|
|
Every format has an identifier of the form `scienting-<name>/<major>`, such as `scienting-doc/1`.
|
|
The identifier tells a reader which rules apply to a file.
|
|
The prefix `scienting-` is the namespace of the formats.
|
|
The namespace is not the name of this repository, paperspec, and formats are named after what they hold.
|
|
|
|
A file or record that can travel on its own carries its identifier in a `format` property.
|
|
The anchor in `fixtures/records/anchor-leaf.json` begins `{"doc":"hale2026harmonic-damb","format":"scienting-anchor/1", ...`.
|
|
Records inside a package file whose format the manifest fixes, such as the lines of `nodes.jsonl`, do not repeat it.
|
|
|
|
A change that makes a valid file invalid, or that changes a computed value, is a breaking change and needs a new major version, with a migration.
|
|
Adding an optional property, a node kind, a run kind, or an enumeration value is a minor change.
|
|
A consumer of version *n* MUST refuse a major version it does not know, and SHOULD warn on, but accept, enumeration values added in a later minor version.
|
|
|
|
The specification as a whole is versioned with semantic versioning; this is version 0.1.0. [registry/formats.json](../registry/formats.json) lists every format with its media type, schema and status.
|
|
|
|
## 0.4 Status
|
|
|
|
Each format has one of three statuses, listed in [registry/formats.json](../registry/formats.json).
|
|
In version 0.1.0 the registry lists 32 formats: 19 are `candidate` and 13 are `provisional`; none is `reserved`.
|
|
|
|
| Status | Meaning |
|
|
| --- | --- |
|
|
| `candidate` | Complete, with vectors and fixtures, and ready to implement. |
|
|
| `provisional` | Complete, but expected to change as implementations gain experience with it. |
|
|
| `reserved` | Named and shaped, but not yet specified in full. |
|
|
|
|
Some fields matter only to some users: a personal library has no use for spot-checks, and a team that reads only open-access papers rarely needs a policy.
|
|
Those fields are defined anyway, so that files move between a personal library and a team's store without conversion.
|
|
A consumer that does not use a field MUST still preserve it when it rewrites a file.
|
|
|
|
## 0.5 How the repository is arranged
|
|
|
|
A normative folder defines what conforming means.
|
|
The other folders help people use the specification.
|
|
|
|
| Path | Holds | Normative |
|
|
| --- | --- | --- |
|
|
| `spec/` | These chapters. | Yes |
|
|
| `schemas/` | JSON Schemas, draft 2020-12. | Yes |
|
|
| `sql/` | Table definitions for SQLite, Dolt and PostgreSQL, and the SQLite bundle. | Yes |
|
|
| `context/` | The JSON-LD context for layers. | Yes |
|
|
| `registry/` | Formats, rule IDs, stamp rules and the default stage requirements. | Yes |
|
|
| `vectors/` | Test vectors, one case per line. | Yes |
|
|
| `fixtures/` | Valid packages and records, invalid packages, and the keys that signed them. | Yes |
|
|
| `tools/` | The programs that compute and check the vectors and fixtures, and load packages into SQLite. | No |
|
|
| `docs/` | Guides for people getting to know the formats. | No |
|
|
|
|
The programs in `tools/` compute every expected value, so that none is typed by hand.
|
|
They are not a library, and implementations conform to the chapters and the vectors.
|