Chemical structures (spec/18 section 18.6) and biological sequences (18.7) tied to
the words of a paper. Minor change: two new body kinds, optional properties and
seven rules, with no valid file made invalid.
- annotation schema: structure (SMILES or SMARTS, role, identity, written, label,
accessions, force-field terms) and sequence (alphabet, role, identity, start,
organism, regions, variants); body_kind in sql/store.sql; nine terms in the
JSON-LD context; rules layer.structure.{value,identity,term} and
layer.sequence.{value,identity,region,variant}.
- paperspec.structures reads the atoms and bonds of a SMILES or SMARTS with the
standard library, so a term's atoms are checked without a toolkit; checked against
RDKit on a corpus and random texts. paperspec.sequences checks alphabets,
identities, regions and variants.
- fixtures/records/chemistry (on the main example) and biology (on the draft, which
gains a last sentence), vectors/structures.jsonl and vectors/sequences.jsonl;
check.py runs both layers and the vectors; counts, guides and CHANGELOG updated.
- the prose check no longer takes organism for a UK spelling.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
62 lines
5.6 KiB
Markdown
62 lines
5.6 KiB
Markdown
# 22. Conformance
|
|
|
|
## 22.1 The suite
|
|
|
|
| Path | Holds | A conforming implementation |
|
|
| --- | --- | --- |
|
|
| `vectors/*.jsonl` | Inputs and expected outputs, one case per line. | Computes every expected output from its input. |
|
|
| `fixtures/packages/` | Valid packages: a converted chemistry paper with corrections, receipts, a multi-level-header table and a dataset; its checks-only export; an open-access paper with two witnesses; a Typst draft; a tiny document, as a folder and as the zip of [12.2](12-package.md#122-paths). | Accepts each, and recomputes every value it contains. |
|
|
| `fixtures/records/` | A valid record of every other format that is a JSON file of its own, and three more layers: traced citations ([18.5](18-layers.md#185-traced-citations)), whose traces resolve as `vectors/traces.jsonl` says, chemical structures ([18.6](18-layers.md#186-chemical-structures)) and sequences ([18.7](18-layers.md#187-sequences)), whose rules `vectors/structures.jsonl` and `vectors/sequences.jsonl` pin; the stage requirements and stamp rules are in `registry/`, and a bundle is built by `paperspec load --bundle`. | Accepts each. |
|
|
| `fixtures/invalid/` | Invalid packages, each with `expect.json` naming the rule it breaks. | Refuses each, and SHOULD name that rule. |
|
|
| `fixtures/keys/` | The test keys that signed the fixtures and the team secret of the checks-only export. | Never uses them for anything else. |
|
|
|
|
All fixture text is synthetic and dedicated to the public domain; license fields inside fixtures describe the fictional works.
|
|
Each part's CI runs this suite against its own implementation, so the parts cannot drift apart silently.
|
|
|
|
## 22.2 Validating a package
|
|
|
|
A validator checks, in order, and reports each failure as `path:line: rule: message`:
|
|
|
|
1. The manifest: present, valid, sorted, every listed file present with its hash and size, nothing unlisted, paths safe and no symbolic links, required files present.
|
|
2. Every file in canonical form, byte for byte, and every record against its schema.
|
|
3. The tree ([3](03-nodes.md)), runs and references between nodes, the counts, and the keys ([2.4](02-model.md#24-work-and-document-keys)).
|
|
4. The policy and hashing scheme ([12.5](12-package.md#125-text-policies)).
|
|
5. The five values of every node, the root and `meta_hash`, when the text is present and the key, if any, is known.
|
|
6. The ID table, the corrections and the history root, the witnesses, pages, locations and references, the formulas derived files name, and every file's order.
|
|
7. Receipts: IDs, roots, documents, node order, signatures against the package's roster, spot-check keys, seeds and samples, and the fidelity distribution.
|
|
|
|
## 22.3 Schemas
|
|
|
|
Schemas use draft 2020-12, restricted to the keywords `$schema`, `$id`, `$ref`, `$defs`, `$comment`, `title`, `description`, `examples`, `default`, `deprecated`, `type`, `enum`, `const`, `properties`, `required`, `additionalProperties`, `patternProperties`, `propertyNames`, `minProperties`, `maxProperties`, `dependentRequired`, `items`, `prefixItems`, `minItems`, `maxItems`, `uniqueItems`, `contains`, `minLength`, `maxLength`, `pattern`, `format`, `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `oneOf`, `anyOf`, `allOf`, `not`, `if`, `then` and `else`.
|
|
Patterns avoid `\d`, `\w`, `\s`, `\b` and lookbehind, whose meaning differs between regular-expression dialects.
|
|
A schema checks shape; the chapters' rules, such as the five values, are checked by validators.
|
|
|
|
## 22.4 Rules
|
|
|
|
[registry/rules.json](../registry/rules.json) lists every rule ID with its meaning.
|
|
Rule IDs are stable: a rule's meaning never changes, and a retired rule's ID is never reused.
|
|
|
|
| Prefix | Covers |
|
|
| --- | --- |
|
|
| `package.` | The manifest, paths, files and schemas. |
|
|
| `doc.tree.`, `doc.node.`, `doc.run.`, `doc.table.`, `doc.math.` | The tree, node content and runs. |
|
|
| `doc.derived.` | Derived files ([12.7](12-package.md#127-derived-files)). |
|
|
| `doc.hash.` | The five values, the root and the metadata hash. |
|
|
| `doc.id.`, `doc.order.` | The ID table and file order. |
|
|
| `doc.policy.` | Text policies and keyed hashing. |
|
|
| `doc.key.`, `doc.counts`, `doc.history` | The keys, the counts and the history root of `document.json`. |
|
|
| `doc.correction.`, `correction.` | A package's corrections, and the gate. |
|
|
| `doc.receipt.`, `doc.qa.`, `doc.fidelity.` | Receipts, spot-checks and levels. |
|
|
| `doc.witness.`, `doc.page.`, `doc.location.`, `doc.reference.` | Witnesses and geometry. |
|
|
| `grade.` | Grading. |
|
|
| `layer.` | Annotations: `layer.trace.source` and `layer.trace.target`, which refer to documents ([18.5](18-layers.md#185-traced-citations)); `layer.structure.value`, `layer.structure.identity` and `layer.structure.term` ([18.6](18-layers.md#186-chemical-structures)); and `layer.sequence.value`, `layer.sequence.identity`, `layer.sequence.region` and `layer.sequence.variant` ([18.7](18-layers.md#187-sequences)). |
|
|
|
|
A package is validated by [22.2](#222-validating-a-package), but a layer is not part of one.
|
|
The `layer.` rules are reported for an annotation, in the form `annotation: rule: message`, by a validator that has the annotation and, for a trace, the states of the documents it names; the reference implementation does so in `paperspec.traces.problems`, `paperspec.structures.problems` and `paperspec.sequences.problems`.
|
|
|
|
## 22.5 Stability
|
|
|
|
Within a major version of a format, a valid file stays valid, and every value the chapters define is computed the same way.
|
|
Rule IDs, enumeration values and format identifiers are never reused with another meaning.
|
|
A `candidate` format changes only with a recorded reason and a new or changed vector; a `provisional` one may change more freely, and says so in [registry/formats.json](../registry/formats.json).
|