Add the data annotation body (spec 18.8)

A data body says what kind of data a table, figure panel or passage presents:
one of fifteen kinds, the dataset that holds the values, what each field
measures (with the dataset's own unit) and the molecules and sequences it is
about, by the identities of structure and sequence bodies. Where the values
came from stays the dataset's origin.

- spec/18 section 18.8, the body in annotation.schema.json, 'data' in the
  store's body_kind CHECK, five terms in the JSON-LD context.
- paperspec.data.problems and the rules layer.data.type, layer.data.quantity,
  layer.data.subject and layer.data.dataset. Units are compared as strings
  with the dataset's; no unit library is needed.
- fixtures/records/data/ (three annotations on the main example) and
  vectors/data.jsonl (35 cases); paperspec check covers both, and the
  determinism rebuild matches.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
This commit is contained in:
alexmandClaude committed 2026-10-18 18:05:11 -04:00
1 parent 3ab96d653f
commit 6025ea3f79
25 files changed
+670 -33

No files matched your search

+1 -1
View File
@@ -428,7 +428,7 @@ Keep using it as a reading and distribution format.
*Status.*
The specification is version 0.1.0.
`registry/formats.json` lists 32 formats: 19 are `candidate` (complete, with vectors and fixtures, ready to implement) and 13 are `provisional` (complete; expected to change as implementations gain experience): project, bundle, dataset, digitization, view, snapshot, draft-ids, exercise, submission, grade, checklist, log-entry and checkpoint ([spec/00 §0.4](../spec/00-introduction.md#04-status)).
The repository holds 24 chapters, 43 JSON Schemas, 28 vector files, five valid fixture packages, one of them also zipped, and 31 invalid ones, each naming the rule it breaks, and a reference checker, `paperspec check`.
The repository holds 24 chapters, 43 JSON Schemas, 29 vector files, five valid fixture packages, one of them also zipped, and 31 invalid ones, each naming the rule it breaks, and a reference checker, `paperspec check`.
Licenses: prose CC BY 4.0; schemas, SQL, the JSON-LD context, registries, vectors and fixtures CC0 1.0; the code in `src/` Apache-2.0 ([README.md](../README.md#licenses)).
*No adoption claim.*
+4 -1
View File
@@ -383,8 +383,11 @@ Nothing finds a molecule in free text, so a person ties each one by hand and the
A structure is a SMILES for a molecule or a SMARTS for a group, a fragment with open positions (`R-OH`) or a pattern of atom types, and it may carry the bonds, angles and torsions the paper gives force-field constants for, citing atoms by their numbers in the text.
A sequence has an alphabet, an optional numbering that starts where the fragment does, regions and variants checked against the residues.
Both carry an `identity` that finds the same structure or sequence however it was written.
A `data` annotation says what kind of data a table, a figure panel or a passage presents, from a fixed list of fifteen kinds such as `binding-affinity`, `energy-profile` or `force-field-parameters`.
It names the dataset that holds the values, what each of its fields measures with the dataset's own unit, and the molecules and sequences the data is about by their identities, so data can be found by its kind and by its subject.
A kind that the list lacks is carried by an outside term in `accessions`, and where the values came from stays the dataset's `origin`.
Layers are `candidate`.
Pinned by the layer records in `fixtures/records/layer/`, `traces/`, `chemistry/` and `biology/`, and by [vectors/traces.jsonl](../vectors/traces.jsonl), [vectors/structures.jsonl](../vectors/structures.jsonl) and [vectors/sequences.jsonl](../vectors/sequences.jsonl).
Pinned by the layer records in `fixtures/records/layer/`, `traces/`, `chemistry/`, `biology/` and `data/`, and by [vectors/traces.jsonl](../vectors/traces.jsonl), [vectors/structures.jsonl](../vectors/structures.jsonl), [vectors/sequences.jsonl](../vectors/sequences.jsonl) and [vectors/data.jsonl](../vectors/data.jsonl).
See [chapter 18](../spec/18-layers.md).
## Exercises
+45
View File
@@ -280,6 +280,51 @@ The identity finds the same thing however it was written: the name a model propo
The last line is the rule that lets a term cite atoms without a chemistry toolkit: `O([H])[H]` writes three atoms, numbered from 0 as it writes them, and the oxygen (0) is bonded to each hydrogen (1 and 2) and the hydrogens are not bonded to each other, so a bond term on atoms 1 and 2, or an angle on atoms 0, 1 and 2, is refused.
[vectors/structures.jsonl](../vectors/structures.jsonl) and [vectors/sequences.jsonl](../vectors/sequences.jsonl) show every rule broken.
## Find data by its kind and by what it is about
This shows how a table or a figure panel is tagged with the kind of data it presents, and how that tag is checked against the dataset and found by a molecule.
A `data` annotation names the dataset that holds the values, the quantities its fields measure, and the molecules and sequences it is about by the identities that `structure` and `sequence` annotations carry ([chapter 18](../spec/18-layers.md#188-data)).
`paperspec.data` checks an annotation against the rules, and against the dataset's descriptor when it is given one, with the standard library alone:
```python
import json
from paperspec import data
descriptors = {}
for folder in ("fixtures/packages/hale2026harmonic-damb/datasets/ds-hale-bond-params",
"fixtures/records/datasets/ds-hale-ctct-scan"):
with open(folder + "/datapackage.json", encoding="utf-8") as f:
d = json.load(f)
descriptors[d["id"]] = d
annotations = [json.loads(line) for line in open("fixtures/records/data/annotations.jsonl", encoding="utf-8")]
for a in annotations:
b = a["body"]
quote = a["target"]["selector"][1]["exact"]
about = [s["label"] for s in b.get("subjects", [])]
print(a["status"], b["role"], b["data_type"], b.get("dataset"), repr(quote), about,
data.problems(a, descriptors.get(b.get("dataset"))))
ethane = "OTMSDBZUPAUEDD-UHFFFAOYSA-N"
for a in annotations:
b = a["body"]
if b["data_type"] == "force-field-parameters" and any(s["identity"] == ethane for s in b.get("subjects", [])):
print([(q["name"], q["unit"]) for q in b["quantities"]])
```
```text
confirmed reports force-field-parameters ds-hale-bond-params 'Fitted harmonic bond parameters.' ['ethane', 'water', 'CT–CT'] []
confirmed reports energy-profile ds-hale-ctct-scan '(a) The scan.' ['ethane'] []
proposed uses force-field-parameters ds-hale-bond-params 'Table 1 lists the fitted constants for both parameter sets.' [] []
[('harmonic bond force constant', 'kcal / mol / angstrom ** 2'), ('equilibrium bond length', 'angstrom')]
```
Each of the first three lines gives the annotation's status, what its span does with the data, the kind, the dataset, the words, the subjects and the rules it breaks, none here.
The caption of Table 1 and the sentence that introduces it lead to one dataset: the caption `reports` it, and the sentence `uses` it and is a model's proposal that a person has not yet confirmed.
The subjects are the identities of the chemistry layer, so a search for force-field parameters about ethane, the last line, finds the table by the molecule and not by the word, and gives the units as the dataset spells them.
The unit is compared as a string and never parsed, so a checker that wants to know that `kcal / mol / angstrom ** 2` is a force constant needs a unit library of its own.
[vectors/data.jsonl](../vectors/data.jsonl) shows every rule broken, among them a quantity whose unit is not its dataset's.
## Test your implementation against a vector
A test vector is a recorded input together with the output every correct implementation must produce.
+1 -1
View File
@@ -107,7 +107,7 @@ A `candidate` format is complete, with vectors and fixtures, and ready to implem
A `provisional` one is complete and expected to change as implementations gain experience.
The provisional formats are dataset, digitization, view, project, bundle, snapshot, draft-ids, exercise, submission, grade, checklist, log-entry and checkpoint.
The repository holds a reference checker (`paperspec check`), 28 vector files, five valid packages, one of them also as a zip, and 31 deliberately broken packages ([spec/00 §0.4](../spec/00-introduction.md#04-status), [spec/22](../spec/22-conformance.md)).
The repository holds a reference checker (`paperspec check`), 29 vector files, five valid packages, one of them also as a zip, and 31 deliberately broken packages ([spec/00 §0.4](../spec/00-introduction.md#04-status), [spec/22](../spec/22-conformance.md)).
## What does it not do?
+4
View File
@@ -44,6 +44,10 @@ Format `scienting-checkpoint/1`, `provisional`. ([23.2](../spec/23-log.md#232-ch
Correction
: An operation on an established document, with its evidence and author; never edited, only appended. ([11](../spec/11-corrections.md))
Data
: What kind of data a table, a figure panel or a passage presents, tied to its words by a `data` annotation: one of fifteen kinds, the dataset that holds the values, what each field measures and the molecules and sequences it is about.
The kind is one reader's reading and a model's stays `proposed`; where the values came from is the dataset's `origin`. ([18.8](../spec/18-layers.md#188-data))
Description
: An annotation that describes a figure, panel or table for readers who cannot see it, as a short text equivalent (`alt`) or a full description (`long`); an accessible export uses confirmed descriptions only. ([18.4](../spec/18-layers.md#184-bodies))
+1 -1
View File
@@ -56,7 +56,7 @@ It appears inside hashed data and test vectors, and it is separate from the repo
| [registry/](../registry/README.md) | The list of formats, the rule IDs a checker reports, download-stamp rules, and default rights requirements. |
| [tools/](../tools/README.md) | How to run the reference commands: `paperspec check` checks everything, `paperspec build` rebuilds the fixtures and vectors, and `paperspec load` loads packages into SQLite. |
| [src/paperspec/](../src/paperspec/) | The oracle, a compact implementation of every rule, which the other repositories of the project install as the Python package `paperspec`. |
| [vectors/](../vectors/README.md) | 28 files of test vectors: inputs and the exact outputs every implementation must compute. |
| [vectors/](../vectors/README.md) | 29 files of test vectors: inputs and the exact outputs every implementation must compute. |
| [fixtures/](../fixtures/README.md) | Five valid example packages, one of them also as a zip, 31 deliberately broken ones that each break one named rule, example records, and the test keys that signed them. All text is synthetic. |
| [sql/](../sql/README.md) | Tables for SQLite, Dolt and PostgreSQL, and a SQLite bundle with full-text search. |
| [context/](../context/README.md) | The JSON-LD context that makes annotation layers W3C Web Annotations. |