feat: structure and sequence annotation bodies; paperspec.structures and paperspec.sequences
Chemical structures (spec/18 section 18.6) and biological sequences (18.7) tied to
the words of a paper. Minor change: two new body kinds, optional properties and
seven rules, with no valid file made invalid.
- annotation schema: structure (SMILES or SMARTS, role, identity, written, label,
accessions, force-field terms) and sequence (alphabet, role, identity, start,
organism, regions, variants); body_kind in sql/store.sql; nine terms in the
JSON-LD context; rules layer.structure.{value,identity,term} and
layer.sequence.{value,identity,region,variant}.
- paperspec.structures reads the atoms and bonds of a SMILES or SMARTS with the
standard library, so a term's atoms are checked without a toolkit; checked against
RDKit on a corpus and random texts. paperspec.sequences checks alphabets,
identities, regions and variants.
- fixtures/records/chemistry (on the main example) and biology (on the draft, which
gains a last sentence), vectors/structures.jsonl and vectors/sequences.jsonl;
check.py runs both layers and the vectors; counts, guides and CHANGELOG updated.
- the prose check no longer takes organism for a UK spelling.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
This commit is contained in:
36 files changed
+1488
-44
No files matched your search
@@ -53,6 +53,8 @@ Replies are annotations that target annotations, so discussions are threads anch
|
||||
| `prerequisite` | `SpecificResource` | `source` | What to read or know first. |
|
||||
| `crosslink` | `SpecificResource` | `source`, `selector` | That the span corresponds to a span in another document. |
|
||||
| `trace` | `SpecificResource` | `source`, `selector`, `reference`, `note`, `verdict`, `proposer` | That the target's sentence cites `reference` on the strength of a passage of the cited document ([18.5](#185-traced-citations)). |
|
||||
| `structure` | `TextualBody` | `format`, `role`, `identity`, `written`, `label`, `accessions`, `terms` | That the span names, draws, gives parameters for or uses a chemical structure ([18.6](#186-chemical-structures)). |
|
||||
| `sequence` | `TextualBody` | `alphabet`, `role`, `identity`, `start`, `label`, `organism`, `accessions`, `regions`, `variants` | That the span names, shows, uses or reports a variant of a protein, DNA or RNA sequence ([18.7](#187-sequences)). |
|
||||
| `translation` | `TextualBody` | `language` | The span in another language. |
|
||||
| `description` | `TextualBody` | `role` | A description of a figure, panel or table for readers who cannot see it: `alt`, a short text equivalent, or `long`, a full description. |
|
||||
| `generated` | `TextualBody` | `generator` | Text a model wrote about the span; never mistaken for source. `generator` is `{model, run}` as a correction's model author is ([11.2](11-corrections.md#112-every-correction-records)). |
|
||||
@@ -132,3 +134,67 @@ A chain of "as previously reported" is then followed one trace at a time: the re
|
||||
A trace breaks `layer.trace.source` when its selector has no `AnchorSelector`, an anchor is in another document than `source` names, `source` is the citing document, or the `TextQuoteSelector` does not repeat the anchor's plain quote.
|
||||
It breaks `layer.trace.target` when the target is not a leaf with runs of the citing document selected by an `AnchorSelector`, or the sentence does not cite `reference` ([22.4](22-conformance.md#224-rules)).
|
||||
`vectors/traces.jsonl` resolves traces against the fixture packages after corrections, with every outcome and every refusal, and `fixtures/records/traces/` holds a layer of four annotations on `writer2027fixed-ynr7.v3` that cites `hale2026harmonic-damb`: a trace a model proposed to the paper's abstract and a person confirmed with a receipt, a candidate a person rejected, a person's trace to Table 1, and a reply that disputes the first.
|
||||
|
||||
## 18.6 Chemical structures
|
||||
|
||||
A paper writes about a molecule in words, and nothing in a converted document finds it there.
|
||||
A `structure` annotation ties one to the words: its target is the span, as for any annotation, and its body holds the structure.
|
||||
The words stay the paper's, the structure is one reader's reading of them, attributed and withdrawable, and a model's tie stays `proposed` until a person confirms it.
|
||||
What a paper reports about a structure, such as the force-field constants of a bond, is about reading this paper ([18.1](#181-one-rule-decides-what-is-a-claim-and-what-is-a-layer)), so it is an annotation here; a claim store holds a parameter once it is checked.
|
||||
|
||||
| Property | Required | Holds |
|
||||
| --- | --- | --- |
|
||||
| `value` | Yes | The structure as a SMILES or a SMARTS, whichever `format` says. |
|
||||
| `format` | Yes | `chemical/x-daylight-smiles` for a SMILES, `chemical/x-daylight-smarts` for a SMARTS. |
|
||||
| `role` | Yes | What the span does with the structure: `names` it, `depicts` it (a figure, scheme or table draws it), `parametrizes` it (the span gives parameters for it) or `uses` it (a reagent, solvent or probe). |
|
||||
| `identity` | Yes | What finds the same structure however it was written: the structure's InChIKey when it is a SMILES and the producer has one, else `smiles:` or `smarts:` and the first twelve hex digits of the SHA-256 of the value as written. |
|
||||
| `written` | No | The structure as the paper writes it when that is not SMILES or SMARTS: `R-OH`, `CT-CT`. |
|
||||
| `label` | No | A short name, such as `NMA`. |
|
||||
| `accessions` | No | Entries of outside databases that the structure is: a list of `{db, id}`, with `db` in lower case (`pubchem`, `chebi`, `cas`). |
|
||||
| `terms` | No | The bonds, angles, torsions and impropers the paper gives values for. |
|
||||
|
||||
A structure that is not one molecule, such as a group, a fragment with open positions or a pattern of atom types, is a SMARTS.
|
||||
An open position is an atom that matches any atom, `[*:1]`, with its label as the atom map number, so `R-C(=O)-R'` is `[*:1]-C(=O)-[*:2]` and `R-OH` is `[*:1]-[OX2H]`, and the paper's own notation goes in `written`.
|
||||
SMILES text that a toolkit would refuse, because a valence rule fails, is kept as a SMARTS rather than dropped, since a paper's drawing often is not a molecule a toolkit can sanitize.
|
||||
|
||||
**Atom numbers.**
|
||||
Atoms are numbered from 0 in the order the value writes them, a bracket atom as one, so `CC(=O)NC` has atoms 0 to 4.
|
||||
A hydrogen that is not written as an atom has no number: a term on a hydrogen needs `[H]` in the value.
|
||||
The rule is of the text and not of any toolkit, which is what lets the numbers be checked without one.
|
||||
|
||||
A term has a `kind` (`bond`, `angle`, `torsion` or `improper`), `atoms` (2, 3, 4 and 4 atom numbers; an improper lists its central atom first), and optionally `types` (the force field's atom types, one for each atom), `values` (the parameters by the name the paper uses, each a decimal string as [1.2](01-conventions.md#12-numbers) writes numbers that carry meaning), `units` (a unit for each value) and `source` (where in the paper).
|
||||
The atoms of a bond, angle or torsion are a chain, each bonded to the next, and the atoms of an improper are each bonded to the first.
|
||||
|
||||
A structure body breaks `layer.structure.value` when its value is not a SMILES or SMARTS as its format says: an unclosed bracket, branch or ring, a stray character, a bond with no atom, or a query (`,`, `;`, `&`, `!` or `$` inside a bracket, or `a` or `A`) in a SMILES.
|
||||
It breaks `layer.structure.identity` when its identity is not of the form its value gives, or, for a `smiles:` or `smarts:` identity, is not the hash of the value.
|
||||
It breaks `layer.structure.term` when a term cites an atom the value does not write or cites an atom twice, when its atoms are not joined as its kind needs, when `types` has another length than `atoms`, or when `units` names a value that is not given ([22.4](22-conformance.md#224-rules)).
|
||||
The reference implementation, `paperspec.structures.problems`, reads atoms and bonds from the text with the standard library alone.
|
||||
It does not check valence, aromaticity or the InChIKey, which need a chemistry toolkit; a producer with one SHOULD also check that the value sanitizes as a molecule when `format` says a SMILES, and that the InChIKey is the molecule's.
|
||||
`vectors/structures.jsonl` reads the atoms and bonds of SMILES and SMARTS texts, mints identities, and breaks every rule, and `fixtures/records/chemistry/` holds a layer of four structures on `hale2026harmonic-damb`: ethane, named and confirmed; water, named by a model and still `proposed`; the CT–CT pattern of Table 1, a SMARTS with the bond constants of both parameter sets as terms; and water again, as a SMILES that writes its hydrogens, with the OW–HW constants.
|
||||
The two writings of water share one InChIKey, which is what finds them as one.
|
||||
|
||||
## 18.7 Sequences
|
||||
|
||||
A `sequence` annotation ties a protein, DNA or RNA sequence to the words that name it, show it or report a variant of it, as [18.6](#186-chemical-structures) does for a structure.
|
||||
|
||||
| Property | Required | Holds |
|
||||
| --- | --- | --- |
|
||||
| `value` | Yes | The residues in capital letters, a gap as `-`. |
|
||||
| `alphabet` | Yes | `protein`, `dna` or `rna`, which says what the letters mean. |
|
||||
| `role` | Yes | `names` the sequence, `depicts` it (the span shows it, as text, a figure or an alignment), `uses` it (a construct, primer or probe) or `reports` something about it (a variant, a region). |
|
||||
| `identity` | Yes | The alphabet, a colon and the first sixteen hex digits of the SHA-256 of the residues with the gaps taken out, so a sequence is found whatever gaps it was shown with and whatever its numbering. |
|
||||
| `start` | No | The number of the first residue of the value in the molecule's own numbering; 1 when omitted. |
|
||||
| `label` | No | A short name, such as `HP35`. |
|
||||
| `organism` | No | The organism the sequence is from, as the paper names it. |
|
||||
| `accessions` | No | Entries of outside databases that the sequence is, as in [18.6](#186-chemical-structures) (`uniprot`, `genbank`, `refseq`, `ensembl`, `pdb`). |
|
||||
| `regions` | No | Stretches of the sequence the span is about: a list of `{start, end, label?}`, both ends included. |
|
||||
| `variants` | No | Changes the span reports: a list of `{position, ref, alt}`. |
|
||||
|
||||
The alphabets allow the twenty amino acids, selenocysteine (`U`), pyrrolysine (`O`), the ambiguity codes `B`, `Z`, `J` and `X` and a stop (`*`) for `protein`; `A`, `C`, `G`, `T` and the IUPAC ambiguity codes for `dna`; the same with `U` for `T` for `rna`; and `-` for a gap in all three.
|
||||
A residue's number is `start` plus the number of residues before it, gaps not counted, so a fragment of a protein is numbered as the whole protein is, and `regions` and `variants` use those numbers.
|
||||
A variant says that the residue at `position` is `ref` in the value and `alt` in the variant; `alt` is a residue, `*` for a stop or `-` for a deletion.
|
||||
|
||||
A sequence body breaks `layer.sequence.value` when its letters are not those of its alphabet or are all gaps, `layer.sequence.identity` when its identity is not the hash of its residues, `layer.sequence.region` when a region starts after it ends or reaches outside the residues, and `layer.sequence.variant` when a variant is outside the residues, names a `ref` the value does not have at that position, changes a residue to itself, or has an `alt` its alphabet does not allow ([22.4](22-conformance.md#224-rules)).
|
||||
The reference implementation is `paperspec.sequences.problems`.
|
||||
It needs no toolkit, since an alphabet is a set of letters; a producer that knows the molecule SHOULD also check the sequence against the `accessions` it names.
|
||||
`vectors/sequences.jsonl` mints identities and breaks every rule, and `fixtures/records/biology/` holds a layer of three annotations on the draft `writer2027fixed-ynr7.v3`, whose last sentence tests its charges on the villin headpiece subdomain HP35: the words that name it, the printed sequence with its three helices as regions, and a model's proposal of the N68H variant, in the numbering 42 to 76 that the headpiece uses.
|
||||
@@ -6,7 +6,7 @@
|
||||
| --- | --- | --- |
|
||||
| `vectors/*.jsonl` | Inputs and expected outputs, one case per line. | Computes every expected output from its input. |
|
||||
| `fixtures/packages/` | Valid packages: a converted chemistry paper with corrections, receipts, a multi-level-header table and a dataset; its checks-only export; an open-access paper with two witnesses; a Typst draft; a tiny document, as a folder and as the zip of [12.2](12-package.md#122-paths). | Accepts each, and recomputes every value it contains. |
|
||||
| `fixtures/records/` | A valid record of every other format that is a JSON file of its own, and a second layer, of traced citations ([18.5](18-layers.md#185-traced-citations)), whose traces resolve as `vectors/traces.jsonl` says; the stage requirements and stamp rules are in `registry/`, and a bundle is built by `paperspec load --bundle`. | Accepts each. |
|
||||
| `fixtures/records/` | A valid record of every other format that is a JSON file of its own, and three more layers: traced citations ([18.5](18-layers.md#185-traced-citations)), whose traces resolve as `vectors/traces.jsonl` says, chemical structures ([18.6](18-layers.md#186-chemical-structures)) and sequences ([18.7](18-layers.md#187-sequences)), whose rules `vectors/structures.jsonl` and `vectors/sequences.jsonl` pin; the stage requirements and stamp rules are in `registry/`, and a bundle is built by `paperspec load --bundle`. | Accepts each. |
|
||||
| `fixtures/invalid/` | Invalid packages, each with `expect.json` naming the rule it breaks. | Refuses each, and SHOULD name that rule. |
|
||||
| `fixtures/keys/` | The test keys that signed the fixtures and the team secret of the checks-only export. | Never uses them for anything else. |
|
||||
|
||||
@@ -49,10 +49,10 @@ Rule IDs are stable: a rule's meaning never changes, and a retired rule's ID is
|
||||
| `doc.receipt.`, `doc.qa.`, `doc.fidelity.` | Receipts, spot-checks and levels. |
|
||||
| `doc.witness.`, `doc.page.`, `doc.location.`, `doc.reference.` | Witnesses and geometry. |
|
||||
| `grade.` | Grading. |
|
||||
| `layer.` | Annotations that refer to documents: `layer.trace.source` and `layer.trace.target` ([18.5](18-layers.md#185-traced-citations)). |
|
||||
| `layer.` | Annotations: `layer.trace.source` and `layer.trace.target`, which refer to documents ([18.5](18-layers.md#185-traced-citations)); `layer.structure.value`, `layer.structure.identity` and `layer.structure.term` ([18.6](18-layers.md#186-chemical-structures)); and `layer.sequence.value`, `layer.sequence.identity`, `layer.sequence.region` and `layer.sequence.variant` ([18.7](18-layers.md#187-sequences)). |
|
||||
|
||||
A package is validated by [22.2](#222-validating-a-package), but a layer is not part of one.
|
||||
The `layer.` rules are reported for an annotation, in the form `annotation: rule: message`, by a validator that has the annotation and the states of the documents it names; the reference implementation does so for traces in `paperspec.traces.problems`.
|
||||
The `layer.` rules are reported for an annotation, in the form `annotation: rule: message`, by a validator that has the annotation and, for a trace, the states of the documents it names; the reference implementation does so in `paperspec.traces.problems`, `paperspec.structures.problems` and `paperspec.sequences.problems`.
|
||||
|
||||
## 22.5 Stability
|
||||
|
||||
|
||||
Reference in new issue
Block a user