Files
paperspec/spec/18-layers.md
T
alexmandClaude 3ab96d653f feat: structure and sequence annotation bodies; paperspec.structures and paperspec.sequences
Chemical structures (spec/18 section 18.6) and biological sequences (18.7) tied to
the words of a paper. Minor change: two new body kinds, optional properties and
seven rules, with no valid file made invalid.

- annotation schema: structure (SMILES or SMARTS, role, identity, written, label,
  accessions, force-field terms) and sequence (alphabet, role, identity, start,
  organism, regions, variants); body_kind in sql/store.sql; nine terms in the
  JSON-LD context; rules layer.structure.{value,identity,term} and
  layer.sequence.{value,identity,region,variant}.
- paperspec.structures reads the atoms and bonds of a SMILES or SMARTS with the
  standard library, so a term's atoms are checked without a toolkit; checked against
  RDKit on a corpus and random texts. paperspec.sequences checks alphabets,
  identities, regions and variants.
- fixtures/records/chemistry (on the main example) and biology (on the draft, which
  gains a last sentence), vectors/structures.jsonl and vectors/sequences.jsonl;
  check.py runs both layers and the vectors; counts, guides and CHANGELOG updated.
- the prose check no longer takes organism for a UK spelling.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
2026-10-18 15:38:31 -04:00

21 KiB
Raw Blame History

18. Layers and annotations

Everything people and models add to a document is an annotation, and annotations are grouped in layers. scienting-layer/1 is the W3C Web Annotation Data Model with the anchor as a selector, so any Web Annotation tool can read it. The JSON-LD context is context/scienting-1.jsonld, published at https://paperspec.scient.ing/context/scienting-1.jsonld.

18.1 One rule decides what is a claim and what is a layer

If a statement is true independently of this paper, it belongs in a claim store as a claim, where it is checked and leveled. If it is about reading this paper, it belongs in a layer, where it is attributed and signed but not checked.

18.2 The layer

A layer is a folder with layer.json and annotations.jsonl. layer.json is an AnnotationCollection with the two contexts, format, an id of the form tag:scient.ing,2026:layer:<store>/<record id>, a label, a purpose, an audience (novice, student, expert or mixed), a visibility (private, team or public), a license (CC BY 4.0 by default when public), its creator, created, its project, the docs it annotates and its total. A reader turns layers on and off, and the audience dial picks the level of glosses.

18.3 The annotation

Each line of annotations.jsonl is a W3C Annotation:

Property Holds
@context ["http://www.w3.org/ns/anno.jsonld", "https://paperspec.scient.ing/context/scienting-1.jsonld"].
id tag:scient.ing,2026:annotation:<store>/<record id> (4.6).
type, layer Annotation, and the layer's ID.
motivation A W3C motivation: commenting, describing, linking, questioning, assessing, replying and the others.
creator, created, modified An agent {id, type, name}, with type Person or Software, and times.
body One body (18.4).
target A SpecificResource whose source is tag:scient.ing,2026:doc:<store>/<key> and whose selector lists an AnchorSelector holding a scienting-anchor/1, optionally followed by a W3C TextQuoteSelector for tools that do not know ours; or, for a reply, a SpecificResource whose source is another annotation.
status proposed, confirmed, rejected or withdrawn. A model's annotation is proposed until a person confirms it.
receipt For a confirmed annotation, the receipt that covers it.

The TextQuoteSelector, when present, repeats the anchor quote's exact, prefix and suffix, with an empty string for a prefix or suffix the quote lacks. It is present only when the anchor's quote is plain text; an anchor whose quote is hashed or keyed (6.3) has none, so the fallback never shows text the anchor withholds.

An annotation's content hash is SHA-256 of the canonical JSON of the annotation without status, receipt and modified. A confirmed annotation is covered by its confirmer's receipt, which lists ["annotation", <record id>, <content hash>]. Replies are annotations that target annotations, so discussions are threads anchored to sentences.

18.4 Bodies

kind W3C type Adds Says
gloss TextualBody audience An explanation at novice, student or expert level.
claim-link SpecificResource source, pin, role That the span states, uses or disputes a claim.
symbol SpecificResource source, pin, key, symbol That this occurrence of a symbol, the one whose key is key, is the symbol a symbol claim defines.
comment TextualBody category A remark, opinion or interpretation.
caveat TextualBody category A warning a reader should see.
question TextualBody for, category A question for a reader, a group or the authors.
exercise TextualBody exercise A task with a key and a rule (19).
citation-context TextualBody reference, verdict, reason Why a sentence cites a reference, and whether it supports it.
prerequisite SpecificResource source What to read or know first.
crosslink SpecificResource source, selector That the span corresponds to a span in another document.
trace SpecificResource source, selector, reference, note, verdict, proposer That the target's sentence cites reference on the strength of a passage of the cited document (18.5).
structure TextualBody format, role, identity, written, label, accessions, terms That the span names, draws, gives parameters for or uses a chemical structure (18.6).
sequence TextualBody alphabet, role, identity, start, label, organism, accessions, regions, variants That the span names, shows, uses or reports a variant of a protein, DNA or RNA sequence (18.7).
translation TextualBody language The span in another language.
description TextualBody role A description of a figure, panel or table for readers who cannot see it: alt, a short text equivalent, or long, a full description.
generated TextualBody generator Text a model wrote about the span; never mistaken for source. generator is {model, run} as a correction's model author is (11.2).

A textual body has a value, and optionally a format (text/plain or text/markdown) and a language.

The source of a claim-link or symbol body is a claim in a paperclaim claim store, written as paperclaim writes a reference: claim: and the claim's tag, eight Crockford base32 capitals (the digits and the letters without I, L, O and U), as in claim:272XTX7C, or claim:<host>/<tag> for a claim in another store, <host> being a domain name. A tag never changes, while a claim's slug, such as amber/k-bond, may be renamed, so a slug is never a reference. The optional pin fixes the claim as the annotator read it: the same reference, @, and the first six lowercase hex digits of the claim's statement hash, as in claim:272XTX7C@02f47b. common.schema.json defines both, as claimRef and claimPin.

A symbol body names its symbol twice, as the target's span and by key, the symbol's key in the math form scienting-math/1 (9.4), which a claim store gives the symbol a symbol claim defines: K_{r} for a span $K_r$. So a reader finds every occurrence of a symbol across a library by its key, however its source spaced or braced it, and a claim store matches the annotation to its claim without parsing the span. The span is still required, since the key says which symbol and the span says where; symbol, optional, is the symbol as the span writes it.

A comment, caveat or question body MAY carry a category, which says what it is about, from a fixed list:

category Says
comment A general remark.
context Background, related work, or what a reader should know to follow the span.
critique A flaw in the method, the analysis or the argument.
question A question.
correction An error of fact, number, citation or reference in the paper.
feedback A suggestion addressed to the authors.
resource Data, code or a related paper.

A producer that supports categories SHOULD set one on every comment, caveat and question; the property is optional only so that layers written before it stay valid. The category is independent of the kind, so a caveat may be a correction and a question a critique. A correction category says that a reader found an error in the paper; correcting the document's reading of it is a correction of 11. The store's annotations table keeps the category beside the body's kind (13.4). Nothing a model writes is ever stored as source text. A model may propose a description, and like any annotation it stays proposed until a person confirms it; an accessible export uses confirmed descriptions only, and a figure with none has no description. Other text a model writes about a span, such as a summary, is a generated annotation, which names its generator. fixtures/records/layer/ holds a layer with one annotation of each common kind, each comment, caveat and question with its category, a claim link and a symbol that reference a paperclaim claim and pin it, and the receipt that confirms some of them.

18.5 Traced citations

"As previously reported [12]": a reader who finds where the cited paper reports it marks the passage, and the trace body keeps the mark. A crosslink says only that two spans correspond, in both directions, whereas a trace goes from one sentence's citation to one passage, and says which reference it follows, why, and who or what found it.

The annotation's target is a leaf or span anchor (6.4) on the citing sentence, which MUST be a leaf with runs, a sentence or a cell, and MUST cite the reference the body names: one of its cite runs lists reference (3.5). reference is the ID of a reference node of the citing document, because a sentence may cite several references at once, as in [3, 12], and the target says only which sentence. source is the cited document, written as a target's source is, tag:scient.ing,2026:doc:<store>/<key>, and never the citing document itself. selector selects the passage in it: an AnchorSelector whose anchor holds the passage's pin (its node_hash) and, for part of a leaf, its span, then a TextQuoteSelector when the anchor's quote is plain (18.3). The passage may be a container, such as a table, whose anchor has no quote.

Property Required Holds
reference Yes The reference node of the citing document that the sentence cites and the trace follows.
source, selector Yes The cited document and the passage in it.
note No What the passage reports, in a person's or a tool's words.
verdict No supports, partly, does-not or cannot-tell, as a citation-context body has it (19.2).
proposer When the creator is Software {model, run, score?, rank?}: the model and run that found the passage as a candidate, as a generated body's generator names them, the candidate's score as a decimal string, and its rank, from 1, among the candidates the run proposed for this citation.

The state of a trace is the annotation's status. A trace a tool proposes is proposed until a person confirms it, and rejected when a person turns the candidate down; a rejected candidate is kept, so that the tool is not offered it again. A trace is disputed while a reply to it, an annotation whose target is the trace and whose motivation is assessing, is proposed or confirmed. The dispute is computed from the replies and never written on the trace, so a reply can be withdrawn.

Whether a trace still leads where it did is also computed, from the current state of both documents. Resolve the target's anchor against the citing document and the body's anchor against the cited document (6.5). The first row that applies gives the outcome:

Outcome When
stale An anchor is stale, on the citing side, the cited side or both.
unlinked The sentence no longer cites reference: a relink removed the citation (11.7), or the reference was deleted.
misresolved A resolve made the reference name a DOI or a document that is not the cited work. A resolution of none names nothing and contradicts nothing (10).
changed Both anchors resolve and at least one is not current.
current Both anchors are current, and the reference still resolves to the cited work or to nothing.

A resolution names the cited work when its document is the cited document, another version of the same work (a key with the same work key, 2.4), or one the caller knows to be the work; and a doi resolution does when the work is known by that DOI (same_as, 2.4). Besides the outcome, a resolution gives the passage's current address, key#node@pin[start:end], while its anchor resolves, and the citations inside the passage. A chain of "as previously reported" is then followed one trace at a time: the reader opens the passage, and a citation in it that has a trace of its own leads on.

A trace breaks layer.trace.source when its selector has no AnchorSelector, an anchor is in another document than source names, source is the citing document, or the TextQuoteSelector does not repeat the anchor's plain quote. It breaks layer.trace.target when the target is not a leaf with runs of the citing document selected by an AnchorSelector, or the sentence does not cite reference (22.4). vectors/traces.jsonl resolves traces against the fixture packages after corrections, with every outcome and every refusal, and fixtures/records/traces/ holds a layer of four annotations on writer2027fixed-ynr7.v3 that cites hale2026harmonic-damb: a trace a model proposed to the paper's abstract and a person confirmed with a receipt, a candidate a person rejected, a person's trace to Table 1, and a reply that disputes the first.

18.6 Chemical structures

A paper writes about a molecule in words, and nothing in a converted document finds it there. A structure annotation ties one to the words: its target is the span, as for any annotation, and its body holds the structure. The words stay the paper's, the structure is one reader's reading of them, attributed and withdrawable, and a model's tie stays proposed until a person confirms it. What a paper reports about a structure, such as the force-field constants of a bond, is about reading this paper (18.1), so it is an annotation here; a claim store holds a parameter once it is checked.

Property Required Holds
value Yes The structure as a SMILES or a SMARTS, whichever format says.
format Yes chemical/x-daylight-smiles for a SMILES, chemical/x-daylight-smarts for a SMARTS.
role Yes What the span does with the structure: names it, depicts it (a figure, scheme or table draws it), parametrizes it (the span gives parameters for it) or uses it (a reagent, solvent or probe).
identity Yes What finds the same structure however it was written: the structure's InChIKey when it is a SMILES and the producer has one, else smiles: or smarts: and the first twelve hex digits of the SHA-256 of the value as written.
written No The structure as the paper writes it when that is not SMILES or SMARTS: R-OH, CT-CT.
label No A short name, such as NMA.
accessions No Entries of outside databases that the structure is: a list of {db, id}, with db in lower case (pubchem, chebi, cas).
terms No The bonds, angles, torsions and impropers the paper gives values for.

A structure that is not one molecule, such as a group, a fragment with open positions or a pattern of atom types, is a SMARTS. An open position is an atom that matches any atom, [*:1], with its label as the atom map number, so R-C(=O)-R' is [*:1]-C(=O)-[*:2] and R-OH is [*:1]-[OX2H], and the paper's own notation goes in written. SMILES text that a toolkit would refuse, because a valence rule fails, is kept as a SMARTS rather than dropped, since a paper's drawing often is not a molecule a toolkit can sanitize.

Atom numbers. Atoms are numbered from 0 in the order the value writes them, a bracket atom as one, so CC(=O)NC has atoms 0 to 4. A hydrogen that is not written as an atom has no number: a term on a hydrogen needs [H] in the value. The rule is of the text and not of any toolkit, which is what lets the numbers be checked without one.

A term has a kind (bond, angle, torsion or improper), atoms (2, 3, 4 and 4 atom numbers; an improper lists its central atom first), and optionally types (the force field's atom types, one for each atom), values (the parameters by the name the paper uses, each a decimal string as 1.2 writes numbers that carry meaning), units (a unit for each value) and source (where in the paper). The atoms of a bond, angle or torsion are a chain, each bonded to the next, and the atoms of an improper are each bonded to the first.

A structure body breaks layer.structure.value when its value is not a SMILES or SMARTS as its format says: an unclosed bracket, branch or ring, a stray character, a bond with no atom, or a query (,, ;, &, ! or $ inside a bracket, or a or A) in a SMILES. It breaks layer.structure.identity when its identity is not of the form its value gives, or, for a smiles: or smarts: identity, is not the hash of the value. It breaks layer.structure.term when a term cites an atom the value does not write or cites an atom twice, when its atoms are not joined as its kind needs, when types has another length than atoms, or when units names a value that is not given (22.4). The reference implementation, paperspec.structures.problems, reads atoms and bonds from the text with the standard library alone. It does not check valence, aromaticity or the InChIKey, which need a chemistry toolkit; a producer with one SHOULD also check that the value sanitizes as a molecule when format says a SMILES, and that the InChIKey is the molecule's. vectors/structures.jsonl reads the atoms and bonds of SMILES and SMARTS texts, mints identities, and breaks every rule, and fixtures/records/chemistry/ holds a layer of four structures on hale2026harmonic-damb: ethane, named and confirmed; water, named by a model and still proposed; the CT–CT pattern of Table 1, a SMARTS with the bond constants of both parameter sets as terms; and water again, as a SMILES that writes its hydrogens, with the OW–HW constants. The two writings of water share one InChIKey, which is what finds them as one.

18.7 Sequences

A sequence annotation ties a protein, DNA or RNA sequence to the words that name it, show it or report a variant of it, as 18.6 does for a structure.

Property Required Holds
value Yes The residues in capital letters, a gap as -.
alphabet Yes protein, dna or rna, which says what the letters mean.
role Yes names the sequence, depicts it (the span shows it, as text, a figure or an alignment), uses it (a construct, primer or probe) or reports something about it (a variant, a region).
identity Yes The alphabet, a colon and the first sixteen hex digits of the SHA-256 of the residues with the gaps taken out, so a sequence is found whatever gaps it was shown with and whatever its numbering.
start No The number of the first residue of the value in the molecule's own numbering; 1 when omitted.
label No A short name, such as HP35.
organism No The organism the sequence is from, as the paper names it.
accessions No Entries of outside databases that the sequence is, as in 18.6 (uniprot, genbank, refseq, ensembl, pdb).
regions No Stretches of the sequence the span is about: a list of {start, end, label?}, both ends included.
variants No Changes the span reports: a list of {position, ref, alt}.

The alphabets allow the twenty amino acids, selenocysteine (U), pyrrolysine (O), the ambiguity codes B, Z, J and X and a stop (*) for protein; A, C, G, T and the IUPAC ambiguity codes for dna; the same with U for T for rna; and - for a gap in all three. A residue's number is start plus the number of residues before it, gaps not counted, so a fragment of a protein is numbered as the whole protein is, and regions and variants use those numbers. A variant says that the residue at position is ref in the value and alt in the variant; alt is a residue, * for a stop or - for a deletion.

A sequence body breaks layer.sequence.value when its letters are not those of its alphabet or are all gaps, layer.sequence.identity when its identity is not the hash of its residues, layer.sequence.region when a region starts after it ends or reaches outside the residues, and layer.sequence.variant when a variant is outside the residues, names a ref the value does not have at that position, changes a residue to itself, or has an alt its alphabet does not allow (22.4). The reference implementation is paperspec.sequences.problems. It needs no toolkit, since an alphabet is a set of letters; a producer that knows the molecule SHOULD also check the sequence against the accessions it names. vectors/sequences.jsonl mints identities and breaks every rule, and fixtures/records/biology/ holds a layer of three annotations on the draft writer2027fixed-ynr7.v3, whose last sentence tests its charges on the villin headpiece subdomain HP35: the words that name it, the printed sequence with its three helices as regions, and a model's proposal of the N68H variant, in the numbering 42 to 76 that the headpiece uses.