alexmandClaude
ea0a7efa78
Features for the next six months (accessible HTML export, paperpack and
Lode on packages, the personal bundle):
- Zipped packages: the oracle writes and reads `.sct.zip` archives under
the rules of spec/12 §12.2; validate.py and load.py accept them; a
byte-identical zip of the tiny package joins the fixtures, and two
broken archives (traversal, symbolic link) join fixtures/invalid.
- The target set of targeted F3 (spec/08 §8.10) is computed by the
oracle and pinned by vectors/targets.jsonl under each digit rule; the
rule now covers cells as well as sentences and the leaves of anchored
containers, and defines a node's section.
- scienting-checklist/1 (provisional): a conformance check of an export,
covered by a receipt as ["checklist", id, hash] (spec/08 §8.11), with a
WCAG 2.1 AA fixture and its receipt.
- QA reports are signed like receipts (spec/08 §8.9).
- A `description` annotation body with role alt or long, so an
accessible export can find a confirmed figure description; the layer
fixture carries one, and the store's body-kind constraint allows it.
- scienting-spot-check/1, the seed payload, is registered with a schema.
- exercise, submission and grade are provisional until a course has
used them.
Audit fixes, each with a vector or a broken package where one was
missing:
- Folder packages: symbolic links are refused on validate and load,
manifest paths must not collide under case folding, listed files are
matched by exact name, files are hashed in chunks under a size limit.
- Files are compared byte for byte, so CR LF, a byte order mark, NaN,
a lone surrogate or invalid UTF-8 is refused instead of translated;
integers beyond 2^53 are read as doubles, as canonical JSON writes them.
- Receipts: signature.key must equal by and the key must be valid when
signed; a qa receipt's nodes must be the sample at the original's
hashes, its scope must name qa.of, its errors must be sampled nodes;
statuses are computed once, iteratively, so chains of spot-checks
hundreds deep settle; Base64 and manifest shape errors become refusals
instead of tracebacks; counts and the fidelity of roster-less packages
are recomputed.
- The gate enforces rule 11 of spec/11 §11.4: changed nodes are
validated against the node schema and the references between nodes
are checked, so a set to a bad value, an insert with unknown
properties or a delete of a referenced header cell is refused; a
relabel to null drops the label; a reclassify drops foreign
attributes; a split strips whitespace across runs; ids.jsonl order
follows spec/04 §4.4 in every state; a move after itself is refused.
- Anchors: the relabel rule requires that nothing else changed below the
node; an alias chain ending in a deletion no longer crashes resolve.
- Notation: a script after an empty group applies to the previous atom
({}^{14}C); letters are defined by Unicode block in spec/05 §5.4 and
the oracle; \left) is complex.
- Page fingerprints delete whitespace before folding, so "10 -20" and
"10-20" fold alike (spec/07 §7.1).
- Grades are summed with exact decimals; the reviewer error bound is
computed with integer arithmetic.
- receipts and revocations are keyed by (doc_key, id) in SQL, since one
revocation may be carried by several packages.
Checking:
- tools/stress.py: seeded randomized property tests over canonical JSON,
normalization, notation, random correction walks, anchor resolution,
Merkle roots, spot-check samples, error bounds, fingerprints, mutated
archives and packages, rights and spot-check chains; CI runs it.
- check.py verifies every count and version the prose states against
the repository, the zipped fixture and crafted archives, the checklist
coverage, and nine more broken packages.
- CONTRIBUTING.md gains a release checklist.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
64 lines
5.1 KiB
Markdown
64 lines
5.1 KiB
Markdown
# 18. Layers and annotations
|
|
|
|
Everything people and models add to a document is an annotation, and annotations are grouped in layers.
|
|
`scienting-layer/1` is the W3C Web Annotation Data Model with the anchor as a selector, so any Web Annotation tool can read it.
|
|
The JSON-LD context is [context/scienting-1.jsonld](../context/scienting-1.jsonld), published at `https://scient.ing/ns/scienting/1.jsonld`.
|
|
|
|
## 18.1 One rule decides what is a claim and what is a layer
|
|
|
|
If a statement is true independently of this paper, it belongs in a claim store as a claim, where it is checked and leveled.
|
|
If it is about reading this paper, it belongs in a layer, where it is attributed and signed but not checked.
|
|
|
|
## 18.2 The layer
|
|
|
|
A layer is a folder with `layer.json` and `annotations.jsonl`.
|
|
`layer.json` is an `AnnotationCollection` with the two contexts, `format`, an `id` of the form `urn:scienting:layer:<store>/<record id>`, a `label`, a `purpose`, an `audience` (`novice`, `student`, `expert` or `mixed`), a `visibility` (`private`, `team` or `public`), a `license` (CC BY 4.0 by default when public), its `creator`, `created`, its `project`, the `docs` it annotates and its `total`.
|
|
A reader turns layers on and off, and the audience dial picks the level of glosses.
|
|
|
|
## 18.3 The annotation
|
|
|
|
Each line of `annotations.jsonl` is a W3C `Annotation`:
|
|
|
|
| Property | Holds |
|
|
| --- | --- |
|
|
| `@context` | `["http://www.w3.org/ns/anno.jsonld", "https://scient.ing/ns/scienting/1.jsonld"]`. |
|
|
| `id` | `urn:scienting:annotation:<store>/<record id>` ([4.6](04-identity.md#46-record-ids)). |
|
|
| `type`, `layer` | `Annotation`, and the layer's ID. |
|
|
| `motivation` | A W3C motivation: `commenting`, `describing`, `linking`, `questioning`, `assessing`, `replying` and the others. |
|
|
| `creator`, `created`, `modified` | An agent `{id, type, name}`, with `type` `Person` or `Software`, and times. |
|
|
| `body` | One body ([18.4](#184-bodies)). |
|
|
| `target` | A `SpecificResource` whose `source` is `urn:scienting:doc:<store>/<key>` and whose `selector` lists an `AnchorSelector` holding a `scienting-anchor/1`, optionally followed by a W3C `TextQuoteSelector` for tools that do not know ours; or, for a reply, a `SpecificResource` whose source is another annotation. |
|
|
| `status` | `proposed`, `confirmed`, `rejected` or `withdrawn`. A model's annotation is `proposed` until a person confirms it. |
|
|
| `receipt` | For a confirmed annotation, the receipt that covers it. |
|
|
|
|
The `TextQuoteSelector`, when present, repeats the anchor quote's `exact`, `prefix` and `suffix`, with an empty string for a `prefix` or `suffix` the quote lacks.
|
|
It is present only when the anchor's quote is plain text; an anchor whose quote is hashed or keyed ([6.3](06-anchors.md#63-keyed-anchors)) has none, so the fallback never shows text the anchor withholds.
|
|
|
|
An annotation's content hash is SHA-256 of the canonical JSON of the annotation without `status`, `receipt` and `modified`.
|
|
A confirmed annotation is covered by its confirmer's receipt, which lists `["annotation", <record id>, <content hash>]`.
|
|
Replies are annotations that target annotations, so discussions are threads anchored to sentences.
|
|
|
|
## 18.4 Bodies
|
|
|
|
| `kind` | W3C type | Adds | Says |
|
|
| --- | --- | --- | --- |
|
|
| `gloss` | `TextualBody` | `audience` | An explanation at `novice`, `student` or `expert` level. |
|
|
| `claim-link` | `SpecificResource` | `source` (`lode:<tag>`), `pin`, `role` | That the span `states`, `uses` or `disputes` a claim. |
|
|
| `symbol` | `SpecificResource` | `source`, `pin`, `symbol` | That this occurrence of a symbol is a symbol claim. |
|
|
| `comment` | `TextualBody` | | A remark, opinion or interpretation. |
|
|
| `caveat` | `TextualBody` | | A warning a reader should see. |
|
|
| `question` | `TextualBody` | `for` | A question for a `reader`, a `group` or the `authors`. |
|
|
| `exercise` | `TextualBody` | `exercise` | A task with a key and a rule ([19](19-exercises.md)). |
|
|
| `citation-context` | `TextualBody` | `reference`, `verdict`, `reason` | Why a sentence cites a reference, and whether it supports it. |
|
|
| `prerequisite` | `SpecificResource` | `source` | What to read or know first. |
|
|
| `crosslink` | `SpecificResource` | `source`, `selector` | That the span corresponds to a span in another document. |
|
|
| `translation` | `TextualBody` | `language` | The span in another language. |
|
|
| `description` | `TextualBody` | `role` | A description of a figure, panel or table for readers who cannot see it: `alt`, a short text equivalent, or `long`, a full description. |
|
|
| `generated` | `TextualBody` | `generator` | Text a model wrote about the span; never mistaken for source. |
|
|
|
|
A textual body has a `value`, and optionally a `format` (`text/plain` or `text/markdown`) and a `language`.
|
|
Nothing a model writes is ever stored as source text.
|
|
A model may propose a `description`, and like any annotation it stays `proposed` until a person confirms it; an accessible export uses confirmed descriptions only, and a figure with none has no description.
|
|
Other text a model writes about a span, such as a summary, is a `generated` annotation, which names its `generator`.
|
|
`fixtures/records/layer/` holds a layer with one annotation of each common kind and the receipt that confirms some of them.
|