The work the other paper tools planned for paperspec (paperdesk PLAN M5.4, paperform PLAN 4.2): - paperspec correct and paperspec.revision apply proposed corrections to a package, folder or zip, and write it only once it validates. - doc.math.form is checked: label and style commands always, the form's own spelling when papermath is installed or forms= is passed. - paperspec.datasets and seven doc.dataset.* rules check a package's datasets. - fixtures/invalid gains suspect-witness and math-form-label (33 in all). - Canonical JSON reads numbers as doubles, so 1e20 round-trips. - The registries and the JSON-LD context ship in the wheel. - Public names for what paperdesk and paperform imported as private ones, with the old names kept as aliases. - The text and lossy-projection functions are about three times faster through the gate, tested equal to the old code on every code point. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013XwDhn4LkrKQfw4ArDEr2E
63 lines
5.9 KiB
Markdown
63 lines
5.9 KiB
Markdown
# 22. Conformance
|
|
|
|
## 22.1 The suite
|
|
|
|
| Path | Holds | A conforming implementation |
|
|
| --- | --- | --- |
|
|
| `vectors/*.jsonl` | Inputs and expected outputs, one case per line. | Computes every expected output from its input. |
|
|
| `fixtures/packages/` | Valid packages: a converted chemistry paper with corrections, receipts, a multi-level-header table and a dataset; its checks-only export; an open-access paper with two witnesses; a Typst draft; a tiny document, as a folder and as the zip of [12.2](12-package.md#122-paths). | Accepts each, and recomputes every value it contains. |
|
|
| `fixtures/records/` | A valid record of every other format that is a JSON file of its own, and four more layers: traced citations ([18.5](18-layers.md#185-traced-citations)), whose traces resolve as `vectors/traces.jsonl` says, chemical structures ([18.6](18-layers.md#186-chemical-structures)), sequences ([18.7](18-layers.md#187-sequences)) and data ([18.8](18-layers.md#188-data)), whose rules `vectors/structures.jsonl`, `vectors/sequences.jsonl` and `vectors/data.jsonl` pin; the stage requirements and stamp rules are in `registry/`, and a bundle is built by `paperspec load --bundle`. | Accepts each. |
|
|
| `fixtures/invalid/` | Invalid packages, each with `expect.json` naming the rule it breaks. | Refuses each, and SHOULD name that rule. |
|
|
| `fixtures/keys/` | The test keys that signed the fixtures and the team secret of the checks-only export. | Never uses them for anything else. |
|
|
|
|
All fixture text is synthetic and dedicated to the public domain; license fields inside fixtures describe the fictional works.
|
|
Each part's CI runs this suite against its own implementation, so the parts cannot drift apart silently.
|
|
|
|
## 22.2 Validating a package
|
|
|
|
A validator checks, in order, and reports each failure as `path:line: rule: message`:
|
|
|
|
1. The manifest: present, valid, sorted, every listed file present with its hash and size, nothing unlisted, paths safe and no symbolic links, required files present.
|
|
2. Every file in canonical form, byte for byte, and every record against its schema.
|
|
3. The tree ([3](03-nodes.md)), runs and references between nodes, the counts, and the keys ([2.4](02-model.md#24-work-and-document-keys)).
|
|
4. The policy and hashing scheme ([12.5](12-package.md#125-text-policies)).
|
|
5. The five values of every node, the root and `meta_hash`, when the text is present and the key, if any, is known.
|
|
6. The ID table, the corrections and the history root, the witnesses, pages, locations and references, the formulas derived files name, and every file's order.
|
|
7. Receipts: IDs, roots, documents, node order, signatures against the package's roster, spot-check keys, seeds and samples, and the fidelity distribution.
|
|
|
|
## 22.3 Schemas
|
|
|
|
Schemas use draft 2020-12, restricted to the keywords `$schema`, `$id`, `$ref`, `$defs`, `$comment`, `title`, `description`, `examples`, `default`, `deprecated`, `type`, `enum`, `const`, `properties`, `required`, `additionalProperties`, `patternProperties`, `propertyNames`, `minProperties`, `maxProperties`, `dependentRequired`, `items`, `prefixItems`, `minItems`, `maxItems`, `uniqueItems`, `contains`, `minLength`, `maxLength`, `pattern`, `format`, `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `oneOf`, `anyOf`, `allOf`, `not`, `if`, `then` and `else`.
|
|
Patterns avoid `\d`, `\w`, `\s`, `\b` and lookbehind, whose meaning differs between regular-expression dialects.
|
|
A schema checks shape; the chapters' rules, such as the five values, are checked by validators.
|
|
|
|
## 22.4 Rules
|
|
|
|
[registry/rules.json](../registry/rules.json) lists every rule ID with its meaning.
|
|
Rule IDs are stable: a rule's meaning never changes, and a retired rule's ID is never reused.
|
|
|
|
| Prefix | Covers |
|
|
| --- | --- |
|
|
| `package.` | The manifest, paths, files and schemas. |
|
|
| `doc.tree.`, `doc.node.`, `doc.run.`, `doc.table.`, `doc.math.` | The tree, node content and runs. |
|
|
| `doc.derived.` | Derived files ([12.7](12-package.md#127-derived-files)). |
|
|
| `doc.dataset.` | Datasets ([15](15-datasets.md)). |
|
|
| `doc.hash.` | The five values, the root and the metadata hash. |
|
|
| `doc.id.`, `doc.order.` | The ID table and file order. |
|
|
| `doc.policy.` | Text policies and keyed hashing. |
|
|
| `doc.key.`, `doc.counts`, `doc.history` | The keys, the counts and the history root of `document.json`. |
|
|
| `doc.correction.`, `correction.` | A package's corrections, and the gate. |
|
|
| `doc.receipt.`, `doc.qa.`, `doc.fidelity.` | Receipts, spot-checks and levels. |
|
|
| `doc.witness.`, `doc.page.`, `doc.location.`, `doc.reference.` | Witnesses and geometry. |
|
|
| `grade.` | Grading. |
|
|
| `layer.` | Annotations: `layer.trace.source` and `layer.trace.target`, which refer to documents ([18.5](18-layers.md#185-traced-citations)); `layer.structure.value`, `layer.structure.identity` and `layer.structure.term` ([18.6](18-layers.md#186-chemical-structures)); `layer.sequence.value`, `layer.sequence.identity`, `layer.sequence.region` and `layer.sequence.variant` ([18.7](18-layers.md#187-sequences)); and `layer.data.type`, `layer.data.quantity`, `layer.data.subject` and `layer.data.dataset` ([18.8](18-layers.md#188-data)). |
|
|
|
|
A package is validated by [22.2](#222-validating-a-package), but a layer is not part of one.
|
|
The `layer.` rules are reported for an annotation, in the form `annotation: rule: message`, by a validator that has the annotation and, for a trace, the states of the documents it names; the reference implementation does so in `paperspec.traces.problems`, `paperspec.structures.problems`, `paperspec.sequences.problems` and `paperspec.data.problems`.
|
|
|
|
## 22.5 Stability
|
|
|
|
Within a major version of a format, a valid file stays valid, and every value the chapters define is computed the same way.
|
|
Rule IDs, enumeration values and format identifiers are never reused with another meaning.
|
|
A `candidate` format changes only with a recorded reason and a new or changed vector; a `provisional` one may change more freely, and says so in [registry/formats.json](../registry/formats.json).
|