feat: validator, bundle builder and release checks for 0.1.0

- Add tools/validate.py: validates packages, and with --correction runs
  proposed corrections through the gate against a package without
  writing anything (spec/11 §11.4). The gate now refuses a proposal that
  fails the correction schema, rule correction.schema.
- tools/load.py validates every package before loading, and with
  --bundle --built-from builds a scienting-bundle/1 with the FTS5 search
  index and bundle_meta (spec/13 §13.5). Search rows are defined per
  unit (sentence, cell, equation outside a sentence, figure-text) with
  the parent kind as context.
- Roster keys carry a role, and a key with the role owner may revoke any
  receipt (spec/08 §8.7, spec/20); roster schema, store table and the
  fixture roster updated. The checker signs a trial revocation to prove
  the rule both ways.
- A qa: receipt raises the nodes it sampled except those in its errors
  (spec/08 §8.2); the example paper's fidelity counts become 27 F3,
  13 F2, 4 F1 and 1 F0.
- The destination chooses the keyed scheme, not the text policy:
  hmac-team between the team's machines, hkdf-pages outside the team
  (spec/05 §5.8, spec/12 §12.5).
- A TextQuoteSelector must repeat the anchor's plain quote and is absent
  for hashed or keyed anchors (spec/18 §18.3).
- Add fixtures/records/suspect.json, a scienting-suspect/1 record.
- Oracle: hashes are kept incrementally and checked after every
  correction against a full recomputation; load and write refuse unsafe
  manifest paths; Ed25519, HKDF and schema validation come from the
  cryptography and jsonschema packages; canonical JSON uses the built-in
  encoder with an exact RFC 8785 fallback.
- The tools need Python 3.14 with cryptography and jsonschema; the py310
  CI environment is dropped and the workflow runs the default only.
- Resolve every [CHECK] note in the docs, READMEs and context; add
  CONTRIBUTING.md; every tool answers --help and refuses unknown options.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
alexmandClaude committed 2026-10-14 18:58:00 -04:00
1 parent b357af84d8
commit 8c8b51cc3f
89 files changed
+1141 -513

No files matched your search

+7 -5
View File
@@ -19,7 +19,8 @@ Two tools that read the same context agree on what `target` or `status` means, e
`scienting-1.jsonld` is the context that makes every annotation layer valid JSON-LD, together with the W3C Web Annotation context.
The Web Annotation context supplies `Annotation`, `body`, `target` and `selector`.
This one supplies the terms the formats add.
It is published at `https://scient.ing/ns/scienting/1.jsonld` [CHECK: the comparison source could not fetch this URL; confirm that it resolves].
Its address is `https://scient.ing/ns/scienting/1.jsonld`.
That address did not resolve when checked on 2026-10-14, so until it does, a JSON-LD processor needs the copy in this folder loaded in its place.
Every layer and every annotation lists both contexts, Web Annotation first:
```json
@@ -65,7 +66,7 @@ A processor reads `layer` as an IRI, and it always reads `docs` as a set, so a o
The file declares `anchor` and `exercise` as JSON literals (`"@type": "@json"`, a JSON-LD 1.1 feature, which is why the file sets `"@version": 1.1`).
A JSON-LD processor then keeps a `scienting-anchor/1` record intact instead of expanding it.
An anchor holds hashes (fixed-length fingerprints of text) and a `span`, a pair of character offsets whose order carries meaning.
Expansion would drop the anchor's unmapped field names and treat its arrays as unordered sets [CHECK: needs citation, JSON-LD 1.1 expansion rules].
Expansion would drop the anchor's unmapped field names ([JSON-LD 1.1 API, expansion step 13.3](https://www.w3.org/TR/json-ld11-api/#alg-expand-each-key-value)) and would give its arrays no order ([JSON-LD 1.1 §4.3](https://www.w3.org/TR/json-ld11/#sets-and-lists)).
## The anchor selector and its fallback
@@ -83,7 +84,7 @@ A tool that does not know `AnchorSelector` cannot use it.
An annotation's `selector` list puts the `AnchorSelector` first and, where the anchor has a quote, a standard `TextQuoteSelector` after it, for tools that do not know ours ([spec/18 §18.3](../spec/18-layers.md#183-the-annotation)).
The formats use no `TextPositionSelector` and no `FragmentSelector`.
Such a tool still reads the annotation's body and its `TextQuoteSelector`; the context matters only to tools that process the data as linked data.
That is the repository's statement; no Web Annotation client has been tested against these files [CHECK: confirm that no client test exists; the comparison source marks this unverified].
That is the repository's statement; no Web Annotation client has been tested against these files.
The fallback is not always there.
Three of the nine fixture annotations have only an `AnchorSelector` (the section-level question, the `generated` annotation on a figure and the exercise on a table), because the anchor points at a container and has no quote.
@@ -104,10 +105,11 @@ The traps:
- `@context` is part of an annotation's content hash (the hash covers everything except `status`, `receipt` and `modified`).
Rewriting or reordering the array changes the hash that a receipt covers.
- `format` is one term for two jobs: the layer's `format` (`scienting-layer/1`) and a text body's `format` (`text/markdown`).
The scienting context maps it to `sct:format` [CHECK: the Web Annotation context defines its own `format`; confirm that listing this context second overrides it].
The scienting context maps it to `sct:format`.
The Web Annotation context maps `format` to `dc:format` and protects none of its terms, and in an array of contexts the later definition wins ([JSON-LD 1.1 §4.1](https://www.w3.org/TR/json-ld11/#advanced-context-usage)), so listing this context second makes `format` mean `sct:format`.
- A consumer that does not use a field must still preserve it when it rewrites a file.
A tool that rewrites layers must not drop the scienting terms ([spec/00 §0.4](../spec/00-introduction.md)).
- The spec does not say what the `TextQuoteSelector` holds when the anchor's quote is hashed (over 200 code points) or keyed ([spec/06 §6.3](../spec/06-anchors.md#63-keyed-anchors)) [CHECK: spec/18 is silent; a plain fallback quote would put text in a public layer that the keyed anchor withholds].
- The `TextQuoteSelector` repeats the anchor's plain quotation, and is left out when the quote is hashed (over 200 code points) or keyed ([spec/18 §18.3](../spec/18-layers.md#183-the-annotation)), so a public layer never shows text that a keyed anchor withholds.
## Status and comparison