feat: structure and sequence annotation bodies; paperspec.structures and paperspec.sequences

Chemical structures (spec/18 section 18.6) and biological sequences (18.7) tied to
the words of a paper. Minor change: two new body kinds, optional properties and
seven rules, with no valid file made invalid.

- annotation schema: structure (SMILES or SMARTS, role, identity, written, label,
  accessions, force-field terms) and sequence (alphabet, role, identity, start,
  organism, regions, variants); body_kind in sql/store.sql; nine terms in the
  JSON-LD context; rules layer.structure.{value,identity,term} and
  layer.sequence.{value,identity,region,variant}.
- paperspec.structures reads the atoms and bonds of a SMILES or SMARTS with the
  standard library, so a term's atoms are checked without a toolkit; checked against
  RDKit on a corpus and random texts. paperspec.sequences checks alphabets,
  identities, regions and variants.
- fixtures/records/chemistry (on the main example) and biology (on the draft, which
  gains a last sentence), vectors/structures.jsonl and vectors/sequences.jsonl;
  check.py runs both layers and the vectors; counts, guides and CHANGELOG updated.
- the prose check no longer takes organism for a UK spelling.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
This commit is contained in:
alexmandClaude committed 2026-10-18 15:38:31 -04:00
1 parent 7890fa1ad7
commit 3ab96d653f
36 files changed
+1488 -44

No files matched your search

+1 -1
View File
@@ -428,7 +428,7 @@ Keep using it as a reading and distribution format.
*Status.*
The specification is version 0.1.0.
`registry/formats.json` lists 32 formats: 19 are `candidate` (complete, with vectors and fixtures, ready to implement) and 13 are `provisional` (complete; expected to change as implementations gain experience): project, bundle, dataset, digitization, view, snapshot, draft-ids, exercise, submission, grade, checklist, log-entry and checkpoint ([spec/00 §0.4](../spec/00-introduction.md#04-status)).
The repository holds 24 chapters, 43 JSON Schemas, 26 vector files, five valid fixture packages, one of them also zipped, and 31 invalid ones, each naming the rule it breaks, and a reference checker, `paperspec check`.
The repository holds 24 chapters, 43 JSON Schemas, 28 vector files, five valid fixture packages, one of them also zipped, and 31 invalid ones, each naming the rule it breaks, and a reference checker, `paperspec check`.
Licenses: prose CC BY 4.0; schemas, SQL, the JSON-LD context, registries, vectors and fixtures CC0 1.0; the code in `src/` Apache-2.0 ([README.md](../README.md#licenses)).
*No adoption claim.*
+6 -1
View File
@@ -378,8 +378,13 @@ Model-written text is always a `generated` annotation, never source, and a model
Whether a generic Web Annotation client displays an `AnchorSelector` has not been tested; the fallback exists so that it can still use the quote.
A `trace` annotation follows a citation to the passage it rests on: it targets the citing sentence and its body names the reference, the cited paper and the passage in it.
Whether a trace still leads there is computed from both papers rather than stored: `current`, `changed`, `stale`, `unlinked` or `misresolved`.
A `structure` annotation ties a chemical structure to the words that name it, and a `sequence` annotation does the same for a protein, DNA or RNA sequence.
Nothing finds a molecule in free text, so a person ties each one by hand and the words stay the paper's.
A structure is a SMILES for a molecule or a SMARTS for a group, a fragment with open positions (`R-OH`) or a pattern of atom types, and it may carry the bonds, angles and torsions the paper gives force-field constants for, citing atoms by their numbers in the text.
A sequence has an alphabet, an optional numbering that starts where the fragment does, regions and variants checked against the residues.
Both carry an `identity` that finds the same structure or sequence however it was written.
Layers are `candidate`.
Pinned by the layer records in `fixtures/records/layer/` and `fixtures/records/traces/`, and by [vectors/traces.jsonl](../vectors/traces.jsonl).
Pinned by the layer records in `fixtures/records/layer/`, `traces/`, `chemistry/` and `biology/`, and by [vectors/traces.jsonl](../vectors/traces.jsonl), [vectors/structures.jsonl](../vectors/structures.jsonl) and [vectors/sequences.jsonl](../vectors/sequences.jsonl).
See [chapter 18](../spec/18-layers.md).
## Exercises
+36
View File
@@ -244,6 +244,42 @@ The first trace is disputed by a reply that says the table is the better passage
The outcomes here are all `current`; [vectors/traces.jsonl](../vectors/traces.jsonl) corrects the draft or the paper and shows the others: `changed` when the passage was reworded but is still found, `stale` when it is gone, `unlinked` when the sentence no longer cites the reference, and `misresolved` when the reference now resolves to another paper.
Resolving a trace also lists the citations inside the passage, so a chain of "as previously reported" is followed one trace at a time.
## Read the structures and sequences tied to a paper's words
This shows how a molecule or a sequence is tied to the words that name it, and how the layer checks what it holds.
Nothing finds a molecule in free text, so a person ties each one: a `structure` annotation holds a SMILES or SMARTS, a `sequence` annotation holds residues, and the span is the annotation's target ([chapter 18](../spec/18-layers.md#186-chemical-structures)).
The main example has a layer of four structures and the draft a layer of three sequences, and `paperspec.structures` and `paperspec.sequences` check each against the rules, with the standard library alone:
```python
import json
from paperspec import sequences, structures
for name, module in (("chemistry", structures), ("biology", sequences)):
for line in open(f"fixtures/records/{name}/annotations.jsonl", encoding="utf-8"):
a = json.loads(line)
quote = a["target"]["selector"][1]["exact"]
print(a["status"], a["body"]["role"], a["body"]["identity"], repr(quote), module.problems(a))
atoms, bonds = structures.scan("O([H])[H]")
print(atoms, sorted(bonds))
```
```text
confirmed names OTMSDBZUPAUEDD-UHFFFAOYSA-N 'ethane' []
proposed names XLYOFNOQVPJJNP-UHFFFAOYSA-N 'Water' []
confirmed parametrizes smarts:2e2a923f840c 'CT–CT' []
confirmed parametrizes XLYOFNOQVPJJNP-UHFFFAOYSA-N 'OW–HW' []
confirmed names protein:5807fcbf6d5567ca 'villin headpiece subdomain HP35' []
confirmed depicts protein:5807fcbf6d5567ca 'LSDEDFKAVFGMTRSAFANLPLWKQQNLKKEKGLF' []
proposed reports protein:5807fcbf6d5567ca 'N68H' []
3 [(0, 1), (0, 2)]
```
Each line gives the annotation's status, what its span does with the structure, the identity, the words, and the rules it breaks, none here.
The identity finds the same thing however it was written: the name a model proposed for water, `O`, and the `O([H])[H]` that Table 1's OW–HW constants are given on are one molecule, and all three annotations of the peptide share one identity.
The last line is the rule that lets a term cite atoms without a chemistry toolkit: `O([H])[H]` writes three atoms, numbered from 0 as it writes them, and the oxygen (0) is bonded to each hydrogen (1 and 2) and the hydrogens are not bonded to each other, so a bond term on atoms 1 and 2, or an angle on atoms 0, 1 and 2, is refused.
[vectors/structures.jsonl](../vectors/structures.jsonl) and [vectors/sequences.jsonl](../vectors/sequences.jsonl) show every rule broken.
## Test your implementation against a vector
A test vector is a recorded input together with the output every correct implementation must produce.
+1 -1
View File
@@ -107,7 +107,7 @@ A `candidate` format is complete, with vectors and fixtures, and ready to implem
A `provisional` one is complete and expected to change as implementations gain experience.
The provisional formats are dataset, digitization, view, project, bundle, snapshot, draft-ids, exercise, submission, grade, checklist, log-entry and checkpoint.
The repository holds a reference checker (`paperspec check`), 26 vector files, five valid packages, one of them also as a zip, and 31 deliberately broken packages ([spec/00 §0.4](../spec/00-introduction.md#04-status), [spec/22](../spec/22-conformance.md)).
The repository holds a reference checker (`paperspec check`), 28 vector files, five valid packages, one of them also as a zip, and 31 deliberately broken packages ([spec/00 §0.4](../spec/00-introduction.md#04-status), [spec/22](../spec/22-conformance.md)).
## What does it not do?
+8
View File
@@ -165,6 +165,10 @@ In the W3C Web Annotation Data Model, the part of an annotation's target that sa
The model defines `TextQuoteSelector` (exact text, with optional prefix and suffix) and `TextPositionSelector` (offsets, which the model calls very brittle), among others ([the W3C model](https://www.w3.org/TR/annotation-model/)).
Layers use a custom `AnchorSelector`, which carries an anchor, followed by a `TextQuoteSelector` for tools that do not know it; they use neither `TextPositionSelector` nor `FragmentSelector`. ([18.3](../spec/18-layers.md#183-the-annotation))
Sequence
: The residues of a protein, DNA or RNA, tied to the words that name, show or report a variant of them by a `sequence` annotation.
Its identity is a hash of the residues with the gaps taken out, and its residues are numbered from the `start` the body gives, so a fragment is numbered as the whole molecule is. ([18.7](../spec/18-layers.md#187-sequences))
Spot-check
: A second reviewer's check of a sample of a receipt's nodes, drawn from a seed nobody can steer: the checker's deterministic signature over the receipt's record hash. ([8.8](../spec/08-fidelity.md#88-spot-checks))
@@ -174,6 +178,10 @@ Stamp rule
Strict projection
: A node's text with notation folded to one canonical form and structure kept; used for certifying agreement. ([5.3](../spec/05-hashes.md#53-the-strict-projection))
Structure
: A chemical structure tied to the words that name, draw, parametrize or use it by a `structure` annotation: a SMILES for a molecule, or a SMARTS for a group, a fragment with open positions or a pattern of atom types.
Atoms are numbered from 0 in the order the text writes them, and the bonds, angles and torsions a paper gives constants for cite them by number. ([18.6](../spec/18-layers.md#186-chemical-structures))
Suspect
: A doubtful reading raised for a person or a review session to answer. ([5.7](../spec/05-hashes.md#57-suspects))
+1 -1
View File
@@ -56,7 +56,7 @@ It appears inside hashed data and test vectors, and it is separate from the repo
| [registry/](../registry/README.md) | The list of formats, the rule IDs a checker reports, download-stamp rules, and default rights requirements. |
| [tools/](../tools/README.md) | How to run the reference commands: `paperspec check` checks everything, `paperspec build` rebuilds the fixtures and vectors, and `paperspec load` loads packages into SQLite. |
| [src/paperspec/](../src/paperspec/) | The oracle, a compact implementation of every rule, which the other repositories of the project install as the Python package `paperspec`. |
| [vectors/](../vectors/README.md) | 26 files of test vectors: inputs and the exact outputs every implementation must compute. |
| [vectors/](../vectors/README.md) | 28 files of test vectors: inputs and the exact outputs every implementation must compute. |
| [fixtures/](../fixtures/README.md) | Five valid example packages, one of them also as a zip, 31 deliberately broken ones that each break one named rule, example records, and the test keys that signed them. All text is synthetic. |
| [sql/](../sql/README.md) | Tables for SQLite, Dolt and PostgreSQL, and a SQLite bundle with full-text search. |
| [context/](../context/README.md) | The JSON-LD context that makes annotation layers W3C Web Annotations. |