Chemical structures (spec/18 section 18.6) and biological sequences (18.7) tied to
the words of a paper. Minor change: two new body kinds, optional properties and
seven rules, with no valid file made invalid.
- annotation schema: structure (SMILES or SMARTS, role, identity, written, label,
accessions, force-field terms) and sequence (alphabet, role, identity, start,
organism, regions, variants); body_kind in sql/store.sql; nine terms in the
JSON-LD context; rules layer.structure.{value,identity,term} and
layer.sequence.{value,identity,region,variant}.
- paperspec.structures reads the atoms and bonds of a SMILES or SMARTS with the
standard library, so a term's atoms are checked without a toolkit; checked against
RDKit on a corpus and random texts. paperspec.sequences checks alphabets,
identities, regions and variants.
- fixtures/records/chemistry (on the main example) and biology (on the draft, which
gains a last sentence), vectors/structures.jsonl and vectors/sequences.jsonl;
check.py runs both layers and the vectors; counts, guides and CHANGELOG updated.
- the prose check no longer takes organism for a UK spelling.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
5.6 KiB
22. Conformance
22.1 The suite
| Path | Holds | A conforming implementation |
|---|---|---|
vectors/*.jsonl |
Inputs and expected outputs, one case per line. | Computes every expected output from its input. |
fixtures/packages/ |
Valid packages: a converted chemistry paper with corrections, receipts, a multi-level-header table and a dataset; its checks-only export; an open-access paper with two witnesses; a Typst draft; a tiny document, as a folder and as the zip of 12.2. | Accepts each, and recomputes every value it contains. |
fixtures/records/ |
A valid record of every other format that is a JSON file of its own, and three more layers: traced citations (18.5), whose traces resolve as vectors/traces.jsonl says, chemical structures (18.6) and sequences (18.7), whose rules vectors/structures.jsonl and vectors/sequences.jsonl pin; the stage requirements and stamp rules are in registry/, and a bundle is built by paperspec load --bundle. |
Accepts each. |
fixtures/invalid/ |
Invalid packages, each with expect.json naming the rule it breaks. |
Refuses each, and SHOULD name that rule. |
fixtures/keys/ |
The test keys that signed the fixtures and the team secret of the checks-only export. | Never uses them for anything else. |
All fixture text is synthetic and dedicated to the public domain; license fields inside fixtures describe the fictional works. Each part's CI runs this suite against its own implementation, so the parts cannot drift apart silently.
22.2 Validating a package
A validator checks, in order, and reports each failure as path:line: rule: message:
- The manifest: present, valid, sorted, every listed file present with its hash and size, nothing unlisted, paths safe and no symbolic links, required files present.
- Every file in canonical form, byte for byte, and every record against its schema.
- The tree (3), runs and references between nodes, the counts, and the keys (2.4).
- The policy and hashing scheme (12.5).
- The five values of every node, the root and
meta_hash, when the text is present and the key, if any, is known. - The ID table, the corrections and the history root, the witnesses, pages, locations and references, the formulas derived files name, and every file's order.
- Receipts: IDs, roots, documents, node order, signatures against the package's roster, spot-check keys, seeds and samples, and the fidelity distribution.
22.3 Schemas
Schemas use draft 2020-12, restricted to the keywords $schema, $id, $ref, $defs, $comment, title, description, examples, default, deprecated, type, enum, const, properties, required, additionalProperties, patternProperties, propertyNames, minProperties, maxProperties, dependentRequired, items, prefixItems, minItems, maxItems, uniqueItems, contains, minLength, maxLength, pattern, format, minimum, maximum, exclusiveMinimum, exclusiveMaximum, oneOf, anyOf, allOf, not, if, then and else.
Patterns avoid \d, \w, \s, \b and lookbehind, whose meaning differs between regular-expression dialects.
A schema checks shape; the chapters' rules, such as the five values, are checked by validators.
22.4 Rules
registry/rules.json lists every rule ID with its meaning. Rule IDs are stable: a rule's meaning never changes, and a retired rule's ID is never reused.
| Prefix | Covers |
|---|---|
package. |
The manifest, paths, files and schemas. |
doc.tree., doc.node., doc.run., doc.table., doc.math. |
The tree, node content and runs. |
doc.derived. |
Derived files (12.7). |
doc.hash. |
The five values, the root and the metadata hash. |
doc.id., doc.order. |
The ID table and file order. |
doc.policy. |
Text policies and keyed hashing. |
doc.key., doc.counts, doc.history |
The keys, the counts and the history root of document.json. |
doc.correction., correction. |
A package's corrections, and the gate. |
doc.receipt., doc.qa., doc.fidelity. |
Receipts, spot-checks and levels. |
doc.witness., doc.page., doc.location., doc.reference. |
Witnesses and geometry. |
grade. |
Grading. |
layer. |
Annotations: layer.trace.source and layer.trace.target, which refer to documents (18.5); layer.structure.value, layer.structure.identity and layer.structure.term (18.6); and layer.sequence.value, layer.sequence.identity, layer.sequence.region and layer.sequence.variant (18.7). |
A package is validated by 22.2, but a layer is not part of one.
The layer. rules are reported for an annotation, in the form annotation: rule: message, by a validator that has the annotation and, for a trace, the states of the documents it names; the reference implementation does so in paperspec.traces.problems, paperspec.structures.problems and paperspec.sequences.problems.
22.5 Stability
Within a major version of a format, a valid file stays valid, and every value the chapters define is computed the same way.
Rule IDs, enumeration values and format identifiers are never reused with another meaning.
A candidate format changes only with a recorded reason and a new or changed vector; a provisional one may change more freely, and says so in registry/formats.json.