Add the data annotation body (spec 18.8)

A data body says what kind of data a table, figure panel or passage presents:
one of fifteen kinds, the dataset that holds the values, what each field
measures (with the dataset's own unit) and the molecules and sequences it is
about, by the identities of structure and sequence bodies. Where the values
came from stays the dataset's origin.

- spec/18 section 18.8, the body in annotation.schema.json, 'data' in the
  store's body_kind CHECK, five terms in the JSON-LD context.
- paperspec.data.problems and the rules layer.data.type, layer.data.quantity,
  layer.data.subject and layer.data.dataset. Units are compared as strings
  with the dataset's; no unit library is needed.
- fixtures/records/data/ (three annotations on the main example) and
  vectors/data.jsonl (35 cases); paperspec check covers both, and the
  determinism rebuild matches.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B8cL4zN4XpaQF5cBtASzfq
This commit is contained in:
alexmandClaude committed 2026-10-18 18:05:11 -04:00
1 parent 3ab96d653f
commit 6025ea3f79
25 files changed
+670 -33

No files matched your search

+6 -4
View File
@@ -52,7 +52,7 @@ The procedure is the same for every file:
3.
Find the inputs and the expected outputs of the case.
The table below says which fields are which, and the chapter it links defines the rule.
Only nine files, `alignment`, `corrections`, `grading`, `resolve`, `rights`, `sequences`, `structures`, `targets` and `traces`, put the expected outputs under `expect`.
Only ten files, `alignment`, `corrections`, `data`, `grading`, `resolve`, `rights`, `sequences`, `structures`, `targets` and `traces`, put the expected outputs under `expect`.
In the other nineteen the expected outputs are fields beside the inputs, such as `output`, `sha256`, `canonical` or `parsed`.
4.
Compute each output from the inputs with your implementation.
@@ -77,15 +77,16 @@ Some files change the procedure:
- `traces.jsonl` names two packages, `citing` (the draft) and `cited` (the main example), and for a `resolve` case lists the corrections to apply to each, in order, before judging the trace, and the `same_as` identifiers of the cited work.
Apply them with the gate ([11.4](../spec/11-corrections.md#114-the-gate)) to the package's state at its own revision.
A `problems` case gives the packages' states as they are, and `cited` is null when the trace names no passage in the main example.
- `structures.jsonl` and `sequences.jsonl` need no package.
- `structures.jsonl`, `sequences.jsonl` and `data.jsonl` need no package.
A `scan` case (structures) gives a SMILES or SMARTS in `text` and its `format`, and expects the number of `atoms` and the `bonds` between them as pairs of atom numbers in order, or null when the text is refused.
An `identity` case expects the identity the body carries, from `text` and `inchikey` (structures) or from `text` and `alphabet` (sequences).
A `problems` case gives a whole annotation and expects the list of rules it breaks, in the order checked; `schema` says whether `schemas/annotation.schema.json` accepts it, because some cases are refused by the schema as well as by a rule.
A `problems` case of `data.jsonl` may also give the `dataset`, the dataset's `datapackage.json` parsed; the rule `layer.data.dataset` then compares the annotation's dataset ID, fields and units with it, and without one it checks only the form of the ID.
Several files also mix more than one kind of case, told apart by their fields.
`ids.jsonl` has minting cases and letter-first cases, `keyed.jsonl` and `signatures.jsonl` have a case for each derivation or record, `spotcheck.jsonl` has sample-size cases and one full sample, and `rights.jsonl` has stage checks (a `stage` field), backend checks (a `backend` field) and one combining case (neither).
The reference checker, `paperspec check` ([tools/](../tools/README.md)), runs all twenty-eight files, together with the rest of the repository's checks:
The reference checker, `paperspec check` ([tools/](../tools/README.md)), runs all twenty-nine files, together with the rest of the repository's checks:
```sh
paperspec check --quick
@@ -130,6 +131,7 @@ The table below uses a few terms that the chapters define:
| `traces.jsonl` | Traced citations: a trace on the draft's sentence that cites the main example, with the corrections that make it current, changed, stale, unlinked or misresolved, the passage's current address and the citations inside it (`op` `resolve`); and traces that break `layer.trace.source` or `layer.trace.target` (`op` `problems`). | [18.5](../spec/18-layers.md#185-traced-citations) |
| `structures.jsonl` | Chemical structures: the atoms and bonds that SMILES and SMARTS texts write, and the texts that are refused (`op` `scan`); the identity of a structure with and without an InChIKey (`op` `identity`); and annotations that are valid or break `layer.structure.value`, `layer.structure.identity` or `layer.structure.term` (`op` `problems`). | [18.6](../spec/18-layers.md#186-chemical-structures) |
| `sequences.jsonl` | Sequences: the identity of a sequence with and without gaps and in each alphabet (`op` `identity`); and annotations that are valid or break `layer.sequence.value`, `layer.sequence.identity`, `layer.sequence.region` or `layer.sequence.variant` (`op` `problems`). | [18.7](../spec/18-layers.md#187-sequences) |
| `data.jsonl` | Data: annotations that are valid, one for each kind of data, or break `layer.data.type`, `layer.data.quantity`, `layer.data.subject` or `layer.data.dataset` (`op` `problems`), some of them checked against the dataset's descriptor. | [18.8](../spec/18-layers.md#188-data) |
| `rights.jsonl` | Stage checks with policies and expiry, the combining rule, and backend conditions. | [14](../spec/14-rights.md) |
| `grading.jsonl` | Exercises, submissions and the grades they earn. | [19.4](../spec/19-exercises.md#194-grading) |
| `targets.jsonl` | The target set of targeted F3 for the example paper under each digit rule, given the anchored nodes. | [8.10](../spec/08-fidelity.md#810-targeted-f3) |
@@ -148,7 +150,7 @@ A candidate format changes only with a recorded reason and a new or changed vect
A passing run shows one thing: for these inputs, your implementation computes the same values as the reference implementation in `src/paperspec/`.
That code computed the expected outputs, and `pixi run build` regenerates them ([tools/](../tools/README.md)).
The vectors hold 454 cases in all, some of them small: `hashes.jsonl` has two documents, `alignment.jsonl` two pairs of readings, `offsets.jsonl` two cases.
The vectors hold 489 cases in all, some of them small: `hashes.jsonl` has two documents, `alignment.jsonl` two pairs of readings, `offsets.jsonl` two cases.
The vectors do not show the following.