feat: structure and sequence annotation bodies; paperspec.structures and paperspec.sequences

Chemical structures (spec/18 section 18.6) and biological sequences (18.7) tied to
the words of a paper. Minor change: two new body kinds, optional properties and
seven rules, with no valid file made invalid.

- annotation schema: structure (SMILES or SMARTS, role, identity, written, label,
  accessions, force-field terms) and sequence (alphabet, role, identity, start,
  organism, regions, variants); body_kind in sql/store.sql; nine terms in the
  JSON-LD context; rules layer.structure.{value,identity,term} and
  layer.sequence.{value,identity,region,variant}.
- paperspec.structures reads the atoms and bonds of a SMILES or SMARTS with the
  standard library, so a term's atoms are checked without a toolkit; checked against
  RDKit on a corpus and random texts. paperspec.sequences checks alphabets,
  identities, regions and variants.
- fixtures/records/chemistry (on the main example) and biology (on the draft, which
  gains a last sentence), vectors/structures.jsonl and vectors/sequences.jsonl;
  check.py runs both layers and the vectors; counts, guides and CHANGELOG updated.
- the prose check no longer takes organism for a UK spelling.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
This commit is contained in:
alexmandClaude committed 2026-10-18 15:38:31 -04:00
1 parent 7890fa1ad7
commit 3ab96d653f
36 files changed
+1488 -44

No files matched your search

+66
View File
@@ -53,6 +53,8 @@ Replies are annotations that target annotations, so discussions are threads anch
| `prerequisite` | `SpecificResource` | `source` | What to read or know first. |
| `crosslink` | `SpecificResource` | `source`, `selector` | That the span corresponds to a span in another document. |
| `trace` | `SpecificResource` | `source`, `selector`, `reference`, `note`, `verdict`, `proposer` | That the target's sentence cites `reference` on the strength of a passage of the cited document ([18.5](#185-traced-citations)). |
| `structure` | `TextualBody` | `format`, `role`, `identity`, `written`, `label`, `accessions`, `terms` | That the span names, draws, gives parameters for or uses a chemical structure ([18.6](#186-chemical-structures)). |
| `sequence` | `TextualBody` | `alphabet`, `role`, `identity`, `start`, `label`, `organism`, `accessions`, `regions`, `variants` | That the span names, shows, uses or reports a variant of a protein, DNA or RNA sequence ([18.7](#187-sequences)). |
| `translation` | `TextualBody` | `language` | The span in another language. |
| `description` | `TextualBody` | `role` | A description of a figure, panel or table for readers who cannot see it: `alt`, a short text equivalent, or `long`, a full description. |
| `generated` | `TextualBody` | `generator` | Text a model wrote about the span; never mistaken for source. `generator` is `{model, run}` as a correction's model author is ([11.2](11-corrections.md#112-every-correction-records)). |
@@ -132,3 +134,67 @@ A chain of "as previously reported" is then followed one trace at a time: the re
A trace breaks `layer.trace.source` when its selector has no `AnchorSelector`, an anchor is in another document than `source` names, `source` is the citing document, or the `TextQuoteSelector` does not repeat the anchor's plain quote.
It breaks `layer.trace.target` when the target is not a leaf with runs of the citing document selected by an `AnchorSelector`, or the sentence does not cite `reference` ([22.4](22-conformance.md#224-rules)).
`vectors/traces.jsonl` resolves traces against the fixture packages after corrections, with every outcome and every refusal, and `fixtures/records/traces/` holds a layer of four annotations on `writer2027fixed-ynr7.v3` that cites `hale2026harmonic-damb`: a trace a model proposed to the paper's abstract and a person confirmed with a receipt, a candidate a person rejected, a person's trace to Table 1, and a reply that disputes the first.
## 18.6 Chemical structures
A paper writes about a molecule in words, and nothing in a converted document finds it there.
A `structure` annotation ties one to the words: its target is the span, as for any annotation, and its body holds the structure.
The words stay the paper's, the structure is one reader's reading of them, attributed and withdrawable, and a model's tie stays `proposed` until a person confirms it.
What a paper reports about a structure, such as the force-field constants of a bond, is about reading this paper ([18.1](#181-one-rule-decides-what-is-a-claim-and-what-is-a-layer)), so it is an annotation here; a claim store holds a parameter once it is checked.
| Property | Required | Holds |
| --- | --- | --- |
| `value` | Yes | The structure as a SMILES or a SMARTS, whichever `format` says. |
| `format` | Yes | `chemical/x-daylight-smiles` for a SMILES, `chemical/x-daylight-smarts` for a SMARTS. |
| `role` | Yes | What the span does with the structure: `names` it, `depicts` it (a figure, scheme or table draws it), `parametrizes` it (the span gives parameters for it) or `uses` it (a reagent, solvent or probe). |
| `identity` | Yes | What finds the same structure however it was written: the structure's InChIKey when it is a SMILES and the producer has one, else `smiles:` or `smarts:` and the first twelve hex digits of the SHA-256 of the value as written. |
| `written` | No | The structure as the paper writes it when that is not SMILES or SMARTS: `R-OH`, `CT-CT`. |
| `label` | No | A short name, such as `NMA`. |
| `accessions` | No | Entries of outside databases that the structure is: a list of `{db, id}`, with `db` in lower case (`pubchem`, `chebi`, `cas`). |
| `terms` | No | The bonds, angles, torsions and impropers the paper gives values for. |
A structure that is not one molecule, such as a group, a fragment with open positions or a pattern of atom types, is a SMARTS.
An open position is an atom that matches any atom, `[*:1]`, with its label as the atom map number, so `R-C(=O)-R'` is `[*:1]-C(=O)-[*:2]` and `R-OH` is `[*:1]-[OX2H]`, and the paper's own notation goes in `written`.
SMILES text that a toolkit would refuse, because a valence rule fails, is kept as a SMARTS rather than dropped, since a paper's drawing often is not a molecule a toolkit can sanitize.
**Atom numbers.**
Atoms are numbered from 0 in the order the value writes them, a bracket atom as one, so `CC(=O)NC` has atoms 0 to 4.
A hydrogen that is not written as an atom has no number: a term on a hydrogen needs `[H]` in the value.
The rule is of the text and not of any toolkit, which is what lets the numbers be checked without one.
A term has a `kind` (`bond`, `angle`, `torsion` or `improper`), `atoms` (2, 3, 4 and 4 atom numbers; an improper lists its central atom first), and optionally `types` (the force field's atom types, one for each atom), `values` (the parameters by the name the paper uses, each a decimal string as [1.2](01-conventions.md#12-numbers) writes numbers that carry meaning), `units` (a unit for each value) and `source` (where in the paper).
The atoms of a bond, angle or torsion are a chain, each bonded to the next, and the atoms of an improper are each bonded to the first.
A structure body breaks `layer.structure.value` when its value is not a SMILES or SMARTS as its format says: an unclosed bracket, branch or ring, a stray character, a bond with no atom, or a query (`,`, `;`, `&`, `!` or `$` inside a bracket, or `a` or `A`) in a SMILES.
It breaks `layer.structure.identity` when its identity is not of the form its value gives, or, for a `smiles:` or `smarts:` identity, is not the hash of the value.
It breaks `layer.structure.term` when a term cites an atom the value does not write or cites an atom twice, when its atoms are not joined as its kind needs, when `types` has another length than `atoms`, or when `units` names a value that is not given ([22.4](22-conformance.md#224-rules)).
The reference implementation, `paperspec.structures.problems`, reads atoms and bonds from the text with the standard library alone.
It does not check valence, aromaticity or the InChIKey, which need a chemistry toolkit; a producer with one SHOULD also check that the value sanitizes as a molecule when `format` says a SMILES, and that the InChIKey is the molecule's.
`vectors/structures.jsonl` reads the atoms and bonds of SMILES and SMARTS texts, mints identities, and breaks every rule, and `fixtures/records/chemistry/` holds a layer of four structures on `hale2026harmonic-damb`: ethane, named and confirmed; water, named by a model and still `proposed`; the CT–CT pattern of Table 1, a SMARTS with the bond constants of both parameter sets as terms; and water again, as a SMILES that writes its hydrogens, with the OW–HW constants.
The two writings of water share one InChIKey, which is what finds them as one.
## 18.7 Sequences
A `sequence` annotation ties a protein, DNA or RNA sequence to the words that name it, show it or report a variant of it, as [18.6](#186-chemical-structures) does for a structure.
| Property | Required | Holds |
| --- | --- | --- |
| `value` | Yes | The residues in capital letters, a gap as `-`. |
| `alphabet` | Yes | `protein`, `dna` or `rna`, which says what the letters mean. |
| `role` | Yes | `names` the sequence, `depicts` it (the span shows it, as text, a figure or an alignment), `uses` it (a construct, primer or probe) or `reports` something about it (a variant, a region). |
| `identity` | Yes | The alphabet, a colon and the first sixteen hex digits of the SHA-256 of the residues with the gaps taken out, so a sequence is found whatever gaps it was shown with and whatever its numbering. |
| `start` | No | The number of the first residue of the value in the molecule's own numbering; 1 when omitted. |
| `label` | No | A short name, such as `HP35`. |
| `organism` | No | The organism the sequence is from, as the paper names it. |
| `accessions` | No | Entries of outside databases that the sequence is, as in [18.6](#186-chemical-structures) (`uniprot`, `genbank`, `refseq`, `ensembl`, `pdb`). |
| `regions` | No | Stretches of the sequence the span is about: a list of `{start, end, label?}`, both ends included. |
| `variants` | No | Changes the span reports: a list of `{position, ref, alt}`. |
The alphabets allow the twenty amino acids, selenocysteine (`U`), pyrrolysine (`O`), the ambiguity codes `B`, `Z`, `J` and `X` and a stop (`*`) for `protein`; `A`, `C`, `G`, `T` and the IUPAC ambiguity codes for `dna`; the same with `U` for `T` for `rna`; and `-` for a gap in all three.
A residue's number is `start` plus the number of residues before it, gaps not counted, so a fragment of a protein is numbered as the whole protein is, and `regions` and `variants` use those numbers.
A variant says that the residue at `position` is `ref` in the value and `alt` in the variant; `alt` is a residue, `*` for a stop or `-` for a deletion.
A sequence body breaks `layer.sequence.value` when its letters are not those of its alphabet or are all gaps, `layer.sequence.identity` when its identity is not the hash of its residues, `layer.sequence.region` when a region starts after it ends or reaches outside the residues, and `layer.sequence.variant` when a variant is outside the residues, names a `ref` the value does not have at that position, changes a residue to itself, or has an `alt` its alphabet does not allow ([22.4](22-conformance.md#224-rules)).
The reference implementation is `paperspec.sequences.problems`.
It needs no toolkit, since an alphabet is a set of letters; a producer that knows the molecule SHOULD also check the sequence against the `accessions` it names.
`vectors/sequences.jsonl` mints identities and breaks every rule, and `fixtures/records/biology/` holds a layer of three annotations on the draft `writer2027fixed-ynr7.v3`, whose last sentence tests its charges on the villin headpiece subdomain HP35: the words that name it, the printed sequence with its three helices as regions, and a model's proposal of the N68H variant, in the numbering 42 to 76 that the headpiece uses.
+3 -3
View File
@@ -6,7 +6,7 @@
| --- | --- | --- |
| `vectors/*.jsonl` | Inputs and expected outputs, one case per line. | Computes every expected output from its input. |
| `fixtures/packages/` | Valid packages: a converted chemistry paper with corrections, receipts, a multi-level-header table and a dataset; its checks-only export; an open-access paper with two witnesses; a Typst draft; a tiny document, as a folder and as the zip of [12.2](12-package.md#122-paths). | Accepts each, and recomputes every value it contains. |
| `fixtures/records/` | A valid record of every other format that is a JSON file of its own, and a second layer, of traced citations ([18.5](18-layers.md#185-traced-citations)), whose traces resolve as `vectors/traces.jsonl` says; the stage requirements and stamp rules are in `registry/`, and a bundle is built by `paperspec load --bundle`. | Accepts each. |
| `fixtures/records/` | A valid record of every other format that is a JSON file of its own, and three more layers: traced citations ([18.5](18-layers.md#185-traced-citations)), whose traces resolve as `vectors/traces.jsonl` says, chemical structures ([18.6](18-layers.md#186-chemical-structures)) and sequences ([18.7](18-layers.md#187-sequences)), whose rules `vectors/structures.jsonl` and `vectors/sequences.jsonl` pin; the stage requirements and stamp rules are in `registry/`, and a bundle is built by `paperspec load --bundle`. | Accepts each. |
| `fixtures/invalid/` | Invalid packages, each with `expect.json` naming the rule it breaks. | Refuses each, and SHOULD name that rule. |
| `fixtures/keys/` | The test keys that signed the fixtures and the team secret of the checks-only export. | Never uses them for anything else. |
@@ -49,10 +49,10 @@ Rule IDs are stable: a rule's meaning never changes, and a retired rule's ID is
| `doc.receipt.`, `doc.qa.`, `doc.fidelity.` | Receipts, spot-checks and levels. |
| `doc.witness.`, `doc.page.`, `doc.location.`, `doc.reference.` | Witnesses and geometry. |
| `grade.` | Grading. |
| `layer.` | Annotations that refer to documents: `layer.trace.source` and `layer.trace.target` ([18.5](18-layers.md#185-traced-citations)). |
| `layer.` | Annotations: `layer.trace.source` and `layer.trace.target`, which refer to documents ([18.5](18-layers.md#185-traced-citations)); `layer.structure.value`, `layer.structure.identity` and `layer.structure.term` ([18.6](18-layers.md#186-chemical-structures)); and `layer.sequence.value`, `layer.sequence.identity`, `layer.sequence.region` and `layer.sequence.variant` ([18.7](18-layers.md#187-sequences)). |
A package is validated by [22.2](#222-validating-a-package), but a layer is not part of one.
The `layer.` rules are reported for an annotation, in the form `annotation: rule: message`, by a validator that has the annotation and the states of the documents it names; the reference implementation does so for traces in `paperspec.traces.problems`.
The `layer.` rules are reported for an annotation, in the form `annotation: rule: message`, by a validator that has the annotation and, for a trace, the states of the documents it names; the reference implementation does so in `paperspec.traces.problems`, `paperspec.structures.problems` and `paperspec.sequences.problems`.
## 22.5 Stability